-- Logs begin at Thu 2022-05-19 20:30:47 UTC, end at Thu 2022-05-19 22:03:29 UTC. -- May 19 21:16:10.675291 dv-cinder-835635-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 19 21:16:10.688135 dv-cinder-835635-2-iscsi devstack@keystone.service[80259]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 19 21:16:10.688135 dv-cinder-835635-2-iscsi devstack@keystone.service[80259]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 19 21:16:10 2022] *** May 19 21:16:10.688135 dv-cinder-835635-2-iscsi devstack@keystone.service[80259]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 19 21:16:10.688135 dv-cinder-835635-2-iscsi devstack@keystone.service[80259]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 19 21:16:10.688135 dv-cinder-835635-2-iscsi devstack@keystone.service[80259]: nodename: dv-cinder-835635-2-iscsi May 19 21:16:10.688135 dv-cinder-835635-2-iscsi devstack@keystone.service[80259]: machine: x86_64 May 19 21:16:10.688135 dv-cinder-835635-2-iscsi devstack@keystone.service[80259]: clock source: unix May 19 21:16:10.688135 dv-cinder-835635-2-iscsi devstack@keystone.service[80259]: pcre jit disabled May 19 21:16:10.688135 dv-cinder-835635-2-iscsi devstack@keystone.service[80259]: detected number of CPU cores: 4 May 19 21:16:10.688135 dv-cinder-835635-2-iscsi devstack@keystone.service[80259]: current working directory: / May 19 21:16:10.688135 dv-cinder-835635-2-iscsi devstack@keystone.service[80259]: detected binary path: /usr/bin/uwsgi-core May 19 21:16:10.688135 dv-cinder-835635-2-iscsi devstack@keystone.service[80259]: your processes number limit is 47680 May 19 21:16:10.688135 dv-cinder-835635-2-iscsi devstack@keystone.service[80259]: your memory page size is 4096 bytes May 19 21:16:10.688135 dv-cinder-835635-2-iscsi devstack@keystone.service[80259]: detected max file descriptor number: 2048 May 19 21:16:10.688135 dv-cinder-835635-2-iscsi devstack@keystone.service[80259]: lock engine: pthread robust mutexes May 19 21:16:10.688135 dv-cinder-835635-2-iscsi devstack@keystone.service[80259]: thunder lock: enabled May 19 21:16:10.688135 dv-cinder-835635-2-iscsi devstack@keystone.service[80259]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 19 21:16:10.688135 dv-cinder-835635-2-iscsi devstack@keystone.service[80259]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 19 21:16:10.732746 dv-cinder-835635-2-iscsi devstack@keystone.service[80259]: Python main interpreter initialized at 0x564fb745b9c0 May 19 21:16:10.732746 dv-cinder-835635-2-iscsi devstack@keystone.service[80259]: python threads support enabled May 19 21:16:10.732746 dv-cinder-835635-2-iscsi devstack@keystone.service[80259]: your server socket listen backlog is limited to 100 connections May 19 21:16:10.732746 dv-cinder-835635-2-iscsi devstack@keystone.service[80259]: your mercy for graceful operations on workers is 90 seconds May 19 21:16:10.733152 dv-cinder-835635-2-iscsi devstack@keystone.service[80259]: mapped 403077 bytes (393 KB) for 2 cores May 19 21:16:10.733152 dv-cinder-835635-2-iscsi devstack@keystone.service[80259]: *** Operational MODE: preforking *** May 19 21:16:10.733469 dv-cinder-835635-2-iscsi devstack@keystone.service[80259]: *** uWSGI is running in multiple interpreter mode *** May 19 21:16:10.733469 dv-cinder-835635-2-iscsi devstack@keystone.service[80259]: spawned uWSGI master process (pid: 80259) May 19 21:16:10.733695 dv-cinder-835635-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 19 21:16:10.734473 dv-cinder-835635-2-iscsi devstack@keystone.service[80259]: spawned uWSGI worker 1 (pid: 80269, cores: 1) May 19 21:16:10.736734 dv-cinder-835635-2-iscsi devstack@keystone.service[80259]: spawned uWSGI worker 2 (pid: 80270, cores: 1) May 19 21:16:10.736734 dv-cinder-835635-2-iscsi devstack@keystone.service[80259]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 19 21:16:16.415270 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 21:16:16.415270 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: from cryptography.utils import int_from_bytes May 19 21:16:16.416140 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 21:16:16.416140 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: from cryptography.utils import int_from_bytes May 19 21:16:16.466185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 21:16:16.466409 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 21:16:16.466660 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] command line args: [] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 21:16:16.466839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 21:16:16.467016 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ================================================================================ {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 21:16:16.467274 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] admin_token = **** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.467522 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] config_dir = [] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.467706 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.467888 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] config_source = [] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.468069 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] debug = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.468275 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.468665 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.468798 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.469059 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.469306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.469534 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] list_limit = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.469728 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] log_config_append = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.469895 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.470058 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] log_dir = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.470288 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] log_file = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.470509 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.470797 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.471001 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.471258 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.471439 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.471594 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.471840 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.472095 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.472314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.472548 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.473078 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.473309 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.473554 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.473795 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.474072 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.474605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.65/identity {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.474787 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] publish_errors = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.475024 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.475258 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.475436 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.475690 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.475925 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.476157 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] standard_threads = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.476385 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.476629 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.476804 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.476979 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] use_journal = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.477153 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] use_json = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.477327 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] use_stderr = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.477501 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] use_syslog = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.477731 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.477986 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.478222 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.478509 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.478743 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.478995 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.479229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.479637 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.479896 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.480144 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.480389 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.480639 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.480866 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.481108 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.481340 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.481568 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.481814 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.482045 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.482322 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.482575 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.482816 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.483034 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.483285 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.483546 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.483798 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.484046 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.484274 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.484503 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.484731 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.484966 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.485211 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.485456 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.485691 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.485918 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.486706 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.486706 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.486706 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.487621 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.487621 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.487621 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.487621 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.487831 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.487975 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.488204 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.488434 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.488669 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.488919 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.489152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.489386 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.489614 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.489846 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.490083 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.490350 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.490580 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.490815 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.491049 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.491283 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.491515 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.491749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.491977 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.492225 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.492459 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.492694 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.492924 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.493184 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.493415 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.493700 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.493873 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.494106 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.494399 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.494638 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.494885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.495544 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.495787 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.496027 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.496265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.496510 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.496746 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.496982 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.497222 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.497457 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.497693 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.497927 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.498161 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.498420 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.498680 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.498916 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.499159 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.499391 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.499630 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.499863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.500101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.500334 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.500563 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.500791 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.501021 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.501253 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.501481 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.501768 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.502005 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.502290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.502518 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.502938 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.503200 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.503442 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.503739 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] auth.external = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.503881 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.504190 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.504424 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.504654 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] auth.password = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.504871 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] auth.token = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.505107 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.505344 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.505594 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.505823 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.506069 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.506325 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.506557 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.506792 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] credential.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.507037 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.507285 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.507535 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.507771 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.508013 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.508247 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.508514 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.508766 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.509001 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.509252 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.509540 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.509786 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.510046 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.510312 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.510597 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.510835 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] federation.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.511070 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.511314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.511729 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.511965 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.512212 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.512449 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.512700 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.512940 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.513189 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.513425 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.513670 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.513905 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.514154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.514428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.514866 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.514993 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.515127 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.515356 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.515591 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.515926 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.516097 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.516328 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.516558 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.516798 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.517031 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.517279 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.517531 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.517777 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.518026 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.518335 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.518561 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.518797 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.519027 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.519288 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.519498 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.519731 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.519973 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.520210 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.520467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.520684 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.520934 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.521175 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.521406 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.521649 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.521894 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.522123 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.522384 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.522615 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.522850 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.523084 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.523330 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.523565 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.523798 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.524045 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.524292 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.524520 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.524746 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.525075 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.525238 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.525470 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.525709 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.525942 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.526203 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.526557 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.526916 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.527148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.527396 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.527640 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.527885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.528118 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.528347 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.528579 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.528811 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.529046 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.529272 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.529515 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.529757 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.530000 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.530260 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.530506 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.530733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.530978 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.531227 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.531442 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.531675 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.531907 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.532150 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.532394 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.532629 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.532879 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.533109 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.533354 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.533591 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.533825 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.534060 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.534338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.534581 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.534811 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.535047 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.535292 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] resource.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.535541 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.535796 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.536027 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.536277 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.536524 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.536758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.537003 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.537236 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.537473 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.537705 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] role.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.537950 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] role.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.538178 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.538452 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.538698 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.538944 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.539188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.539431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.539675 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.539916 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.540161 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.540390 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.540633 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.540878 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.541123 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.541365 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.542005 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.542126 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.542517 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.542768 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.543015 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.543263 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.543496 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.543750 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.544000 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.544237 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.544468 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.544699 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.544930 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.545177 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.545429 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.545674 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.545916 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.546150 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.546405 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.546641 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] token.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.546875 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.547121 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.547357 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.547610 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.547856 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.548092 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.548334 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.548574 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.548817 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.549051 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.549285 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.549517 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.549758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.550005 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.550232 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.550494 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.550788 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.551045 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.connection = **** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.551286 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.551531 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.551766 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.552003 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.552236 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.552468 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.552699 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.552934 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.553174 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.553405 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.553646 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.553880 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.554123 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.554382 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.554625 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.554851 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.555088 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.555319 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.555487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 21:16:16.587654 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:16.587856 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:16.588085 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:16.597058 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:16.625531 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:16.625769 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:16.678555 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:16.732245 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:16.732465 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:16.799597 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:16.799797 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:16.800027 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:16.800258 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:16.800472 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:16.800668 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:16.824150 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:16.824370 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:16.824604 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:16.824801 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:16.825038 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:16.825268 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:16.825457 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:16.837316 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:16.852579 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:16.852789 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:16.853016 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:16.853266 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:16.853456 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:16.853674 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:16.853916 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:16.871762 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.871929 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 21:16:16.872292 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.872474 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 21:16:16.872769 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.873027 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.873283 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.873533 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.884598 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.884840 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.885125 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.885403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.893609 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:16:16.896931 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:16:16.897216 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.897486 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.897879 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.898164 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.898421 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.898713 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.898982 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.899248 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.899519 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.899764 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.926388 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.928027 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:16:16.928304 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.929289 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 21:16:16.929289 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: from cryptography.utils import int_from_bytes May 19 21:16:16.929821 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 21:16:16.929821 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: from cryptography.utils import int_from_bytes May 19 21:16:16.933656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:16:16.933937 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.934209 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.941582 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:16:16.941863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.946516 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:16:16.946786 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.947056 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.947356 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.947628 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.947902 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.948207 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.948456 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.967142 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.967418 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.972982 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:16:16.976289 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 21:16:16.976456 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 21:16:16.976647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.976777 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] command line args: [] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 21:16:16.976891 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 21:16:16.976990 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.977068 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ================================================================================ {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 21:16:16.977328 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] admin_token = **** {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.977569 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] config_dir = [] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.977724 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.977929 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] config_source = [] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.978078 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] debug = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.978347 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.978550 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.978752 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.978997 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.979234 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.979492 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] list_limit = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.979664 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] log_config_append = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.979830 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.980007 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] log_dir = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.980179 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] log_file = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.980406 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.980647 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.980869 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.981109 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.981283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.981455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.981692 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.981930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.982118 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:16:16.982200 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.982397 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.982629 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.982855 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.983093 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.983331 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.983617 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.984110 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.65/identity {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.984286 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] publish_errors = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.984512 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.984736 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.984908 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.985146 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.985374 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.985601 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] standard_threads = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.985726 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:16:16.985826 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.986061 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.986232 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.986430 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] use_journal = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.986602 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] use_json = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.986771 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] use_stderr = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.986942 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] use_syslog = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.987167 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 21:16:16.987415 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.987645 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.987900 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.988132 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.988372 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.988602 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.988842 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.989071 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.989310 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.989471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.989552 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.989765 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.989845 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.989993 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.990080 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.990232 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.990351 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.990483 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.990649 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.990742 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.990948 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.991027 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:16.991178 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.991425 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.991671 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.991907 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.992136 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.992386 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.992639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.992896 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.993141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.993368 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.993594 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.993819 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.994051 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.994324 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.994576 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.994804 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.995425 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.995662 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.995991 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.996144 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.996382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.996606 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.996836 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.997064 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.997288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.997513 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.997738 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.997961 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.998191 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.998458 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.998684 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.998914 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.999138 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.999367 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.999595 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:16.999835 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.000059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.000291 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.000521 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.000749 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.000978 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.001207 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.001429 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.001671 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.001908 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.002138 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.002390 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.002643 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.002872 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.003098 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.003330 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.003558 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.003819 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.004049 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.004290 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.004545 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.004776 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.005006 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.005237 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.005479 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.005711 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.005942 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.006178 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.006433 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.006664 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.006893 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.007121 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.007350 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.007603 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.007836 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.008069 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.008300 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.008535 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.008766 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.008995 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.009224 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.009448 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.009672 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.009895 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.010122 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.010366 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.010449 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:16:17.010706 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.010850 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.011050 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.011146 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.011254 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.011369 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.011449 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.011696 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.011949 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.012185 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.012415 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] auth.external = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.012639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.013033 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.013264 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.013491 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] auth.password = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.013717 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] auth.token = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.013954 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.014188 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.014456 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.014684 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.014926 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.015168 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.015394 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.015623 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] credential.caching = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.015862 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.016105 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.016345 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.016575 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.016813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.017042 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.017301 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.017551 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.017785 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.018049 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.018359 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.018602 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.018859 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.019094 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.019300 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.019394 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.019557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] federation.caching = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.019789 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.020025 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.020264 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.020486 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.020725 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.020955 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.021034 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.021200 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.021280 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.021429 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.021678 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.021918 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.022161 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.022416 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.caching = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.022661 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.022901 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.023133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.023373 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.023616 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.023839 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.024070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.024315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.024396 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:16:17.024559 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.024638 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.024784 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.024917 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.025011 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.025239 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.025328 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.025449 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.025528 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.025688 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.025927 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.026210 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.026528 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.026823 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.027025 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.027259 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.027485 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.027717 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.027941 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.028171 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.028404 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.028636 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.028878 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.029103 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.029343 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.029581 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.029810 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.030057 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.030322 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.030549 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.030779 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.031000 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.031231 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.031459 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.031691 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.031919 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.032148 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.032393 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.032631 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.032856 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.033077 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.033318 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.033557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.033785 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.034013 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.034267 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.034495 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.034728 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.034992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.035217 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.035476 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.035720 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.035961 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.036191 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.036415 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.036645 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.036872 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.037103 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.037326 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.037566 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.037806 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.038046 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.038305 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.038551 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.038777 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.038904 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:16:17.039002 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.039183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.039266 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.039447 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.039528 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.039674 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.039753 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.039903 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.039990 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.040142 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.040253 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.040383 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.040525 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.040604 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.040848 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.041071 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.041308 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.041536 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.041789 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.042159 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.042409 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.042645 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.042873 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.043105 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.043340 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] resource.caching = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.043583 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.043823 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.044053 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.044296 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.044610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.044844 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.045083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.045315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.045548 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.045776 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] role.caching = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.046016 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] role.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.046415 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.046861 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.047293 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.047741 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.048098 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.048361 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.048609 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.048854 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.049152 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.049578 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.049894 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.050143 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.050415 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.050664 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.051072 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.051305 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.051648 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.052139 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.052569 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.053026 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.053452 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.053807 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.054063 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.054344 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.054579 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.054811 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.055041 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.055290 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.055543 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.055788 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.056101 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.056536 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.056931 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.057301 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] token.caching = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.057710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.058116 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.058420 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.058675 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.058921 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.059046 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:16:17.059174 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.059385 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.059633 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.059869 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.060101 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.060347 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.060579 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.060822 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.061065 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.061289 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.061527 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.061821 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.062082 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.connection = **** {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.062281 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:16:17.062405 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.062613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.062810 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.063049 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.063281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.063511 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.063741 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.063972 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.064223 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.064502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.064962 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.065419 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.065603 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.065884 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.066145 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.066421 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.066654 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.066894 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.067198 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 21:16:17.067277 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.067512 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 21:16:17.068864 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:16:17.069143 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.069427 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.076940 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.078589 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:16:17.081804 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.082092 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.082400 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.082684 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.092165 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:16:17.092525 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.092852 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.095406 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:17.095648 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:17.095872 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:17.100822 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:16:17.101104 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.101388 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.101657 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.101932 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.102219 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.102517 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.102784 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.103057 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.103321 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.103577 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.103837 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.104119 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.104474 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.105975 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:17.124709 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:17.124923 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:17.143896 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x564fb745b9c0 pid: 80269 (default app) May 19 21:16:17.147338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701da874-085e-4516-a387-5127b3edbc83 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:17.147644 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701da874-085e-4516-a387-5127b3edbc83 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:17.147927 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701da874-085e-4516-a387-5127b3edbc83 None None] PATH_INFO: `/v3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:17.152413 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1/1] 10.222.0.65 () {50 vars in 791 bytes} [Thu May 19 21:16:17 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:17.161540 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:17.199511 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:17.199680 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:17.241997 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:17.242199 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:17.242466 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:17.242641 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:17.242830 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:17.243056 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:17.259294 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:17.259504 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:17.259754 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:17.259955 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:17.260275 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:17.260467 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:17.262914 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:17.274417 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:17.274417 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:17.274417 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:17.274417 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:17.274417 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:17.274417 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:17.277000 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:17.277000 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 21:16:17.287835 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.288009 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 21:16:17.288375 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.288539 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 21:16:17.289609 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.289844 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.290292 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.290469 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.303940 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.304215 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.304475 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.304756 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.313196 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:16:17.316539 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:16:17.316813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.317087 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.317361 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.317745 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.318013 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.318296 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.318573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.318869 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.319107 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.319397 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.346070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.347744 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:16:17.348020 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.353277 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:16:17.353584 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.353833 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.361340 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:16:17.361623 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.366315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:16:17.366597 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.366871 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.367147 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.367431 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.367682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.367954 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.368218 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.388337 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.388618 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.393154 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:16:17.396784 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.397071 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.402409 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:16:17.405987 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:16:17.409708 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.409999 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.410280 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.410580 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.410849 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.411134 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.430131 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:16:17.430553 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.430820 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.431080 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.440413 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.442135 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.442439 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.445469 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:16:17.445746 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.446025 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.446298 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.446588 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.460183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:16:17.460365 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.460657 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.460926 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.461213 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.461477 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.461744 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.480771 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:16:17.484147 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:16:17.487368 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.488940 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.490592 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:16:17.490872 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.491148 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.499669 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.501337 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:16:17.504673 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.504973 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.505252 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.505532 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.515640 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:16:17.515929 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.516207 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.524658 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 21:16:17.524940 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.525213 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.525482 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.525757 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.526028 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.526400 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.526670 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.526945 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.527209 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.527471 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.527731 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.527986 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.528252 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 21:16:17.568119 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x564fb745b9c0 pid: 80270 (default app) May 19 21:16:25.698705 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf35f50-9b5b-4b35-b128-414982329fe6 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:25.699409 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf35f50-9b5b-4b35-b128-414982329fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:25.699409 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf35f50-9b5b-4b35-b128-414982329fe6 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:25.700594 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 2/2] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:16:25.708677 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab5a375-752d-4b1d-bb6b-3b9a57831c17 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:25.709270 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab5a375-752d-4b1d-bb6b-3b9a57831c17 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:25.709270 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab5a375-752d-4b1d-bb6b-3b9a57831c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:25.776593 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: INFO dbcounter [None req-6ab5a375-752d-4b1d-bb6b-3b9a57831c17 None None] Registered counter for database keystone May 19 21:16:25.799482 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG oslo_db.sqlalchemy.engines [None req-6ab5a375-752d-4b1d-bb6b-3b9a57831c17 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80270) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 21:16:25.806733 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writer thread running {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 21:16:25.992031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG passlib.handlers.bcrypt [None req-6ab5a375-752d-4b1d-bb6b-3b9a57831c17 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80270) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 21:16:25.992400 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG passlib.handlers.bcrypt [None req-6ab5a375-752d-4b1d-bb6b-3b9a57831c17 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80270) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 21:16:26.049307 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-6ab5a375-752d-4b1d-bb6b-3b9a57831c17 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:16:26.050968 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6ab5a375-752d-4b1d-bb6b-3b9a57831c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:16:26.183303 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1/3] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:16:25 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 478 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 21:16:26.370913 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf35f50-9b5b-4b35-b128-414982329fe6 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:26.440766 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: INFO dbcounter [None req-baf35f50-9b5b-4b35-b128-414982329fe6 None None] Registered counter for database keystone May 19 21:16:26.461111 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG oslo_db.sqlalchemy.engines [None req-baf35f50-9b5b-4b35-b128-414982329fe6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80269) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 21:16:26.469162 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writer thread running {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 21:16:26.592048 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf35f50-9b5b-4b35-b128-414982329fe6 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:26.596969 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59f56b-66e4-44da-8b38-e09a78cfd89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:26.597654 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59f56b-66e4-44da-8b38-e09a78cfd89a admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:26.597944 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59f56b-66e4-44da-8b38-e09a78cfd89a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:26.598205 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59f56b-66e4-44da-8b38-e09a78cfd89a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:26.607853 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c59f56b-66e4-44da-8b38-e09a78cfd89a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:26.659875 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7c59f56b-66e4-44da-8b38-e09a78cfd89a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 21:16:26.660165 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c59f56b-66e4-44da-8b38-e09a78cfd89a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:26.671034 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 3/4] 10.222.0.65 () {58 vars in 1324 bytes} [Thu May 19 21:16:26 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 21:16:26.677324 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab5a375-752d-4b1d-bb6b-3b9a57831c17 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:26.693616 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab5a375-752d-4b1d-bb6b-3b9a57831c17 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:26.699751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f329b16-f3a4-4ad4-a04b-f9c52b33688c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:26.700939 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f329b16-f3a4-4ad4-a04b-f9c52b33688c admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:26.701227 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f329b16-f3a4-4ad4-a04b-f9c52b33688c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:26.701506 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f329b16-f3a4-4ad4-a04b-f9c52b33688c admin admin] PATH_INFO: `/v3/projects/0436212ebbfa4ccb91d5457faaa851ee` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:26.703328 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f329b16-f3a4-4ad4-a04b-f9c52b33688c admin admin] RBAC: Authorizing `identity:get_project(project_id=0436212ebbfa4ccb91d5457faaa851ee)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:26.757883 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8f329b16-f3a4-4ad4-a04b-f9c52b33688c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 21:16:26.758593 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f329b16-f3a4-4ad4-a04b-f9c52b33688c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:26.763944 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 2/5] 10.222.0.65 () {56 vars in 1207 bytes} [Thu May 19 21:16:26 2022] GET /identity/v3/projects/0436212ebbfa4ccb91d5457faaa851ee => generated 344 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:28.979805 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee49d43-ec3b-46c1-ad40-22963a4bd85e None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:28.980388 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee49d43-ec3b-46c1-ad40-22963a4bd85e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:28.980388 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee49d43-ec3b-46c1-ad40-22963a4bd85e None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:28.984593 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 4/6] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:16:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:16:28.990699 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a3fd24-781a-4c01-afbd-c7d2294101e7 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:28.991249 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a3fd24-781a-4c01-afbd-c7d2294101e7 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:28.991249 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a3fd24-781a-4c01-afbd-c7d2294101e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:29.011643 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b4a3fd24-781a-4c01-afbd-c7d2294101e7 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:16:29.013226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b4a3fd24-781a-4c01-afbd-c7d2294101e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:16:29.021267 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 3/7] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:16:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 21:16:29.044488 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee49d43-ec3b-46c1-ad40-22963a4bd85e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:29.058551 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee49d43-ec3b-46c1-ad40-22963a4bd85e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:29.063210 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8160322-82d2-4035-a5da-55190996afd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:29.063878 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8160322-82d2-4035-a5da-55190996afd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:29.064169 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8160322-82d2-4035-a5da-55190996afd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:29.064452 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8160322-82d2-4035-a5da-55190996afd7 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:29.073401 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8160322-82d2-4035-a5da-55190996afd7 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:29.075282 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8160322-82d2-4035-a5da-55190996afd7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:29.084984 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 5/8] 10.222.0.65 () {58 vars in 1324 bytes} [Thu May 19 21:16:29 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 19 21:16:29.091260 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a3fd24-781a-4c01-afbd-c7d2294101e7 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:29.101271 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a3fd24-781a-4c01-afbd-c7d2294101e7 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:29.107033 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c76a9-59d5-4722-8387-79064845f8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:29.107929 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c76a9-59d5-4722-8387-79064845f8e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:29.108222 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c76a9-59d5-4722-8387-79064845f8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:29.108499 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c76a9-59d5-4722-8387-79064845f8e2 admin admin] PATH_INFO: `/v3/users/d45bbb3805754099870efea6f5d53dfe` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:29.110085 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c76a9-59d5-4722-8387-79064845f8e2 admin admin] RBAC: Authorizing `identity:get_user(user_id=d45bbb3805754099870efea6f5d53dfe)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:29.112009 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c76a9-59d5-4722-8387-79064845f8e2 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:29.113658 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 4/9] 10.222.0.65 () {56 vars in 1198 bytes} [Thu May 19 21:16:29 2022] GET /identity/v3/users/d45bbb3805754099870efea6f5d53dfe => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:31.436424 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc94ed-61ca-4122-8138-2424beca3b08 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:31.437075 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc94ed-61ca-4122-8138-2424beca3b08 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:31.437075 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc94ed-61ca-4122-8138-2424beca3b08 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:31.438361 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 6/10] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:16:31.444183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf6fbb-ef84-4f7f-9412-182dccd3b73c None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:31.444740 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf6fbb-ef84-4f7f-9412-182dccd3b73c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:31.444867 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf6fbb-ef84-4f7f-9412-182dccd3b73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:31.463942 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-9cdf6fbb-ef84-4f7f-9412-182dccd3b73c None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:16:31.465150 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9cdf6fbb-ef84-4f7f-9412-182dccd3b73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:16:31.486785 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c58bf34-6901-44c8-8f8a-4eb3d14e0bd9 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:31.487076 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c58bf34-6901-44c8-8f8a-4eb3d14e0bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:31.487357 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c58bf34-6901-44c8-8f8a-4eb3d14e0bd9 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:31.488807 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 7/11] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:16:31.497870 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e3ffb-740b-453d-affa-1497b02b7439 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:31.498139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e3ffb-740b-453d-affa-1497b02b7439 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:31.498471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e3ffb-740b-453d-affa-1497b02b7439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:31.540529 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG passlib.handlers.bcrypt [None req-dc5e3ffb-740b-453d-affa-1497b02b7439 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80269) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 21:16:31.540983 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG passlib.handlers.bcrypt [None req-dc5e3ffb-740b-453d-affa-1497b02b7439 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80269) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 21:16:31.561372 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 5/12] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:16:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:16:31.588791 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-dc5e3ffb-740b-453d-affa-1497b02b7439 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:16:31.589504 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf6fbb-ef84-4f7f-9412-182dccd3b73c None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:31.590586 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-dc5e3ffb-740b-453d-affa-1497b02b7439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:16:31.631405 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:31.698462 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 8/13] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:16:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:16:31.720936 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e3ffb-740b-453d-affa-1497b02b7439 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:31.735559 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf6fbb-ef84-4f7f-9412-182dccd3b73c None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:31.777258 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec47dad-841c-4800-ba85-2f5444f0b645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:31.778392 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec47dad-841c-4800-ba85-2f5444f0b645 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:31.778945 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec47dad-841c-4800-ba85-2f5444f0b645 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:31.779310 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec47dad-841c-4800-ba85-2f5444f0b645 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:31.793856 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec47dad-841c-4800-ba85-2f5444f0b645 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:16:31.805697 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-dec47dad-841c-4800-ba85-2f5444f0b645 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:16:31.805697 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:16:31.805697 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:16:31.805697 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:16:31.805697 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:16:31.805697 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:16:31.805697 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:16:31.805697 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:16:31.805697 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:16:31.805697 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:16:31.805697 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:16:31.805697 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:16:31.805697 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:16:31.805697 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:16:31.805697 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:16:31.805697 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:16:31.807766 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:16:31.807766 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:16:31.807766 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:16:31.807766 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:16:31.807766 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:16:31.807766 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:16:31.807766 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:16:31.807766 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:16:31.807766 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:16:31.807766 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:16:31.807766 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:16:31.807766 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:16:31.807766 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:16:31.807766 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:16:31.807766 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:16:31.807766 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:16:31.808813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:16:31.808813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:16:31.808813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:16:31.808813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:16:31.808813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:16:31.808813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:16:31.808813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:16:31.808813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:16:31.808813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:16:31.808813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:16:31.808813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:16:31.808813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:16:31.808813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 6/14] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 21:16:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:16:31.813530 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec47dad-841c-4800-ba85-2f5444f0b645 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:31.948459 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e3ffb-740b-453d-affa-1497b02b7439 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:31.951516 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:32.008335 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6705eba9-9a59-400d-b62d-01f98acc92be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:32.009278 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6705eba9-9a59-400d-b62d-01f98acc92be None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:32.009765 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6705eba9-9a59-400d-b62d-01f98acc92be None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:32.010052 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6705eba9-9a59-400d-b62d-01f98acc92be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:32.051426 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:32.071738 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec47dad-841c-4800-ba85-2f5444f0b645 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:32.092803 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6705eba9-9a59-400d-b62d-01f98acc92be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:32.094412 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6705eba9-9a59-400d-b62d-01f98acc92be None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:32.111187 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3488cd7-7dd6-45bd-8249-9476172c4da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:32.112020 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3488cd7-7dd6-45bd-8249-9476172c4da8 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:32.112417 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3488cd7-7dd6-45bd-8249-9476172c4da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:32.112772 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3488cd7-7dd6-45bd-8249-9476172c4da8 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:32.113702 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3488cd7-7dd6-45bd-8249-9476172c4da8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:16:32.114108 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3488cd7-7dd6-45bd-8249-9476172c4da8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:32.115702 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3488cd7-7dd6-45bd-8249-9476172c4da8 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:32.130156 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 7/15] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:16:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:32.139036 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3488cd7-7dd6-45bd-8249-9476172c4da8 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:32.220296 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 9/16] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:16:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 21:16:32.226712 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6705eba9-9a59-400d-b62d-01f98acc92be None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:32.231332 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:32.275443 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3488cd7-7dd6-45bd-8249-9476172c4da8 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:32.315438 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db778841-0d02-4353-8c18-9fd4cd9fe2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:32.316419 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db778841-0d02-4353-8c18-9fd4cd9fe2df None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:32.316765 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db778841-0d02-4353-8c18-9fd4cd9fe2df None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:32.317109 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db778841-0d02-4353-8c18-9fd4cd9fe2df None admin] PATH_INFO: `/v3/users/d45bbb3805754099870efea6f5d53dfe` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:32.318506 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db778841-0d02-4353-8c18-9fd4cd9fe2df None admin] RBAC: Authorizing `identity:get_user(user_id=d45bbb3805754099870efea6f5d53dfe)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:32.320419 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db778841-0d02-4353-8c18-9fd4cd9fe2df None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:32.322121 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 8/17] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:16:32 2022] GET /identity/v3/users/d45bbb3805754099870efea6f5d53dfe => generated 247 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:16:32.328059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db778841-0d02-4353-8c18-9fd4cd9fe2df None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:32.345452 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:32.348574 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6705eba9-9a59-400d-b62d-01f98acc92be None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:32.390323 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e4b9a-024b-4115-b45f-c436800f5f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:32.391151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e4b9a-024b-4115-b45f-c436800f5f7a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:32.391430 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e4b9a-024b-4115-b45f-c436800f5f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:32.391701 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e4b9a-024b-4115-b45f-c436800f5f7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:32.412326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356e4b9a-024b-4115-b45f-c436800f5f7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:32.413890 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356e4b9a-024b-4115-b45f-c436800f5f7a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:32.436629 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-356e4b9a-024b-4115-b45f-c436800f5f7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:16:32.436629 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:16:32.436629 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:16:32.436629 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:16:32.436629 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:16:32.436629 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:16:32.436629 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:16:32.436629 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:16:32.436629 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:16:32.436629 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:16:32.436629 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:16:32.436629 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:16:32.436629 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:16:32.436629 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:16:32.436629 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:16:32.438749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:16:32.438749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:16:32.438749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:16:32.438749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:16:32.438749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:16:32.438749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:16:32.438749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:16:32.438749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:16:32.438749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:16:32.438749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:16:32.438749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:16:32.438749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:16:32.438749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:16:32.438749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:16:32.438749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:16:32.438749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:16:32.438749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:16:32.440408 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:16:32.440408 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:16:32.440408 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:16:32.440408 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:16:32.440408 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 10/18] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:16:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:16:32.442827 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db778841-0d02-4353-8c18-9fd4cd9fe2df None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:32.444035 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e4b9a-024b-4115-b45f-c436800f5f7a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:32.479260 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76c79d8-e91b-44d2-9807-549e1dd8a4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:32.480222 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c79d8-e91b-44d2-9807-549e1dd8a4ac None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:32.480501 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c79d8-e91b-44d2-9807-549e1dd8a4ac None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:32.480776 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c79d8-e91b-44d2-9807-549e1dd8a4ac None admin] PATH_INFO: `/v3/domains/default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:32.482018 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76c79d8-e91b-44d2-9807-549e1dd8a4ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:32.483536 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76c79d8-e91b-44d2-9807-549e1dd8a4ac None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:32.485171 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 9/19] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:16:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:16:32.495024 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76c79d8-e91b-44d2-9807-549e1dd8a4ac None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:32.513900 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:32.569186 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e4b9a-024b-4115-b45f-c436800f5f7a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:32.583708 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:32.607029 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76c79d8-e91b-44d2-9807-549e1dd8a4ac None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:32.608764 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29737cdf-0c62-452e-9159-754a19adf8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:32.609593 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29737cdf-0c62-452e-9159-754a19adf8ee None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:32.609997 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29737cdf-0c62-452e-9159-754a19adf8ee None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:32.610423 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29737cdf-0c62-452e-9159-754a19adf8ee None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:32.611259 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29737cdf-0c62-452e-9159-754a19adf8ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:16:32.611651 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29737cdf-0c62-452e-9159-754a19adf8ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:32.613153 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29737cdf-0c62-452e-9159-754a19adf8ee None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:32.634947 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 11/20] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:16:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:16:32.647482 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4247590d-9d1f-4db0-898d-eacc5641e7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:32.648200 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4247590d-9d1f-4db0-898d-eacc5641e7a6 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:32.648482 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4247590d-9d1f-4db0-898d-eacc5641e7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:32.648763 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4247590d-9d1f-4db0-898d-eacc5641e7a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:32.649648 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4247590d-9d1f-4db0-898d-eacc5641e7a6 None admin] RBAC: Adding query filter params (role.id=2b714f2e5c6c469f8fa037bda1ed16f6, scope.domain.id=default, user.id=d45bbb3805754099870efea6f5d53dfe) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:16:32.649958 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4247590d-9d1f-4db0-898d-eacc5641e7a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:32.651619 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4247590d-9d1f-4db0-898d-eacc5641e7a6 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:32.659978 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 10/21] 10.222.0.65 () {56 vars in 1322 bytes} [Thu May 19 21:16:32 2022] GET /identity/v3/role_assignments?user.id=d45bbb3805754099870efea6f5d53dfe&scope.domain.id=default&role.id=2b714f2e5c6c469f8fa037bda1ed16f6 => generated 233 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:16:35.287554 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce9ad3-a6e4-4cd6-be30-3acf9078338e None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:35.288528 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce9ad3-a6e4-4cd6-be30-3acf9078338e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:35.289004 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce9ad3-a6e4-4cd6-be30-3acf9078338e None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:35.290666 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 12/22] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:16:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:16:35.296876 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad12353a-437f-43af-9e17-19ff530a94d0 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:35.297810 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad12353a-437f-43af-9e17-19ff530a94d0 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:35.298326 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad12353a-437f-43af-9e17-19ff530a94d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:35.317049 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-ad12353a-437f-43af-9e17-19ff530a94d0 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:16:35.318619 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ad12353a-437f-43af-9e17-19ff530a94d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:16:35.393961 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 11/23] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:16:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:16:35.477672 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ce9ad3-a6e4-4cd6-be30-3acf9078338e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:35.628005 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ce9ad3-a6e4-4cd6-be30-3acf9078338e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:35.651415 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:35.679939 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8ed733-663f-40ee-b05d-d573be5bd7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:35.681235 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ed733-663f-40ee-b05d-d573be5bd7a7 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:35.681738 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ed733-663f-40ee-b05d-d573be5bd7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:35.682417 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ed733-663f-40ee-b05d-d573be5bd7a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:35.704466 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8ed733-663f-40ee-b05d-d573be5bd7a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:35.705969 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8ed733-663f-40ee-b05d-d573be5bd7a7 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:35.734093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-dd8ed733-663f-40ee-b05d-d573be5bd7a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:16:35.734093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:16:35.734093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:16:35.734093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:16:35.734093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:16:35.734093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:16:35.734093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:16:35.734093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:16:35.734093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:16:35.734093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:16:35.734093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:16:35.734093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:16:35.734093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:16:35.734093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:16:35.734093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:16:35.734991 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:16:35.734991 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:16:35.734991 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:16:35.734991 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:16:35.734991 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:16:35.734991 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:16:35.734991 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:16:35.734991 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:16:35.734991 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:16:35.734991 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:16:35.734991 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:16:35.734991 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:16:35.734991 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:16:35.734991 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:16:35.734991 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:16:35.734991 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:16:35.734991 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:16:35.735777 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:16:35.735777 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:16:35.735777 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:16:35.735777 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:16:35.735777 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 13/24] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:16:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:16:35.741288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad12353a-437f-43af-9e17-19ff530a94d0 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:35.938021 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:35.975501 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad12353a-437f-43af-9e17-19ff530a94d0 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:36.036024 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31774bc-e1e4-4208-bdd1-3dbfc7f4d837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:36.036763 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31774bc-e1e4-4208-bdd1-3dbfc7f4d837 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:36.037049 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31774bc-e1e4-4208-bdd1-3dbfc7f4d837 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:36.037330 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31774bc-e1e4-4208-bdd1-3dbfc7f4d837 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:36.038123 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31774bc-e1e4-4208-bdd1-3dbfc7f4d837 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:16:36.038460 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31774bc-e1e4-4208-bdd1-3dbfc7f4d837 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:36.039880 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31774bc-e1e4-4208-bdd1-3dbfc7f4d837 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:36.069906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 12/25] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:16:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:36.076182 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8ed733-663f-40ee-b05d-d573be5bd7a7 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:36.148686 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb86303-1abf-4f85-8c71-37c565906c0a None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:36.149002 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb86303-1abf-4f85-8c71-37c565906c0a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:36.149251 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb86303-1abf-4f85-8c71-37c565906c0a None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:36.151051 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 13/26] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:16:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:16:36.158041 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167348fd-5054-4b56-8fbc-2ab972624825 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:36.158371 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167348fd-5054-4b56-8fbc-2ab972624825 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:36.158649 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167348fd-5054-4b56-8fbc-2ab972624825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:36.189376 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-167348fd-5054-4b56-8fbc-2ab972624825 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:16:36.190555 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-167348fd-5054-4b56-8fbc-2ab972624825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:16:36.289448 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8ed733-663f-40ee-b05d-d573be5bd7a7 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:36.346073 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e0ec90-a784-449b-907c-cc0f1dacc0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:36.347848 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0ec90-a784-449b-907c-cc0f1dacc0df None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:36.348139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0ec90-a784-449b-907c-cc0f1dacc0df None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:36.348424 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0ec90-a784-449b-907c-cc0f1dacc0df None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:36.349405 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e0ec90-a784-449b-907c-cc0f1dacc0df None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:36.350950 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e0ec90-a784-449b-907c-cc0f1dacc0df None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:36.365447 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 14/27] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:16:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:16:36.366654 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f6b71-0221-454f-bf50-f2670eff019d None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:36.366946 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f6b71-0221-454f-bf50-f2670eff019d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:36.367228 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f6b71-0221-454f-bf50-f2670eff019d None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:36.375935 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 15/28] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:16:36 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:16:36.381658 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1e7f7-c4f2-45ff-a5ed-79a55580950b None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:36.381947 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1e7f7-c4f2-45ff-a5ed-79a55580950b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:36.382229 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1e7f7-c4f2-45ff-a5ed-79a55580950b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:36.414119 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-e8d1e7f7-c4f2-45ff-a5ed-79a55580950b None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:16:36.415520 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e8d1e7f7-c4f2-45ff-a5ed-79a55580950b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:16:36.538756 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:36.553866 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 16/29] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:16:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:16:36.555923 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1e7f7-c4f2-45ff-a5ed-79a55580950b None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:36.561538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 14/30] 10.222.0.65 () {60 vars in 1141 bytes} [Thu May 19 21:16:36 2022] POST /identity/v3/projects => generated 301 bytes in 486 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:16:36.594849 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e0ec90-a784-449b-907c-cc0f1dacc0df None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:36.644689 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:36.792703 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1e7f7-c4f2-45ff-a5ed-79a55580950b None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:36.840824 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4983de-806a-4c3b-94c3-e586f73471ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:36.841519 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4983de-806a-4c3b-94c3-e586f73471ec None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:36.841802 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4983de-806a-4c3b-94c3-e586f73471ec None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:36.842080 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4983de-806a-4c3b-94c3-e586f73471ec None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:36.842889 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4983de-806a-4c3b-94c3-e586f73471ec None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:36.844320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4983de-806a-4c3b-94c3-e586f73471ec None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:36.861195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e0ec90-a784-449b-907c-cc0f1dacc0df None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:36.922321 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913a6079-03c9-4d18-8bfb-71fca14f7354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:36.923717 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a6079-03c9-4d18-8bfb-71fca14f7354 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:36.924209 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a6079-03c9-4d18-8bfb-71fca14f7354 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:36.924769 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a6079-03c9-4d18-8bfb-71fca14f7354 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:36.950305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913a6079-03c9-4d18-8bfb-71fca14f7354 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:16:36.969865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-913a6079-03c9-4d18-8bfb-71fca14f7354 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:16:36.969865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:16:36.969865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:16:36.969865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:16:36.969865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:16:36.969865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:16:36.969865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:16:36.969865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:16:36.969865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:16:36.969865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:16:36.969865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:16:36.969865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:16:36.969865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:16:36.969865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:16:36.969865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:16:36.969865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:16:36.970791 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:16:36.970791 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:16:36.970791 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:16:36.970791 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:16:36.970791 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:16:36.970791 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:16:36.970791 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:16:36.970791 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:16:36.970791 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:16:36.970791 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:16:36.970791 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:16:36.970791 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:16:36.970791 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:16:36.970791 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:16:36.970791 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:16:36.970791 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:16:36.971543 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:16:36.971543 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:16:36.971543 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:16:36.971543 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:16:36.971543 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:16:36.971543 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:16:36.971543 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:16:36.971543 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:16:36.971543 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:16:36.971543 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:16:36.971543 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:16:36.971543 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:16:36.972423 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 15/31] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 21:16:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:16:36.975294 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6343d76-a212-437e-ba6d-f54eb853c8dc None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:36.975790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6343d76-a212-437e-ba6d-f54eb853c8dc None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:36.976211 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6343d76-a212-437e-ba6d-f54eb853c8dc None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:36.981030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 17/33] 10.222.0.65 () {60 vars in 1131 bytes} [Thu May 19 21:16:36 2022] POST /identity/v3/roles => generated 219 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:16:36.981706 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4983de-806a-4c3b-94c3-e586f73471ec None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:36.981905 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 16/33] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:16:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:16:36.991858 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffde093-716d-4b80-8c24-3e2f311a6eb8 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:36.992707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffde093-716d-4b80-8c24-3e2f311a6eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:36.993582 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffde093-716d-4b80-8c24-3e2f311a6eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:37.039802 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-8ffde093-716d-4b80-8c24-3e2f311a6eb8 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:16:37.042576 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8ffde093-716d-4b80-8c24-3e2f311a6eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:16:37.105667 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:37.206687 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4983de-806a-4c3b-94c3-e586f73471ec None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:37.247974 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 17/34] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:16:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:16:37.284690 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6cd1ce-839b-40ea-a991-e233ffeb6f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:37.286799 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6cd1ce-839b-40ea-a991-e233ffeb6f13 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:37.287533 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffde093-716d-4b80-8c24-3e2f311a6eb8 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:37.287799 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6cd1ce-839b-40ea-a991-e233ffeb6f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:37.288465 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6cd1ce-839b-40ea-a991-e233ffeb6f13 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:37.290207 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6cd1ce-839b-40ea-a991-e233ffeb6f13 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:16:37.290959 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6cd1ce-839b-40ea-a991-e233ffeb6f13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:37.294922 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6cd1ce-839b-40ea-a991-e233ffeb6f13 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:37.312199 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:37.315582 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 18/35] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:16:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:37.324871 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6cd1ce-839b-40ea-a991-e233ffeb6f13 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:37.495665 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6cd1ce-839b-40ea-a991-e233ffeb6f13 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:37.514268 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:37.533006 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b1cf9-c97d-44fd-8b57-2ece9fd29ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:37.533897 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b1cf9-c97d-44fd-8b57-2ece9fd29ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:37.534181 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b1cf9-c97d-44fd-8b57-2ece9fd29ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:37.534489 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b1cf9-c97d-44fd-8b57-2ece9fd29ca2 None admin] PATH_INFO: `/v3/users/d45bbb3805754099870efea6f5d53dfe` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:37.535788 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314b1cf9-c97d-44fd-8b57-2ece9fd29ca2 None admin] RBAC: Authorizing `identity:get_user(user_id=d45bbb3805754099870efea6f5d53dfe)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:37.537246 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314b1cf9-c97d-44fd-8b57-2ece9fd29ca2 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:37.537914 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffde093-716d-4b80-8c24-3e2f311a6eb8 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:37.543851 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 19/36] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:16:37 2022] GET /identity/v3/users/d45bbb3805754099870efea6f5d53dfe => generated 247 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:16:37.543851 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543bf87-7b4b-486a-958a-c77ec4c08533 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:37.543851 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543bf87-7b4b-486a-958a-c77ec4c08533 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:37.543851 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543bf87-7b4b-486a-958a-c77ec4c08533 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:37.543851 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 20/37] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:16:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:16:37.543851 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c3187-f79f-423c-96d2-2b224b6a0b70 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:37.544479 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c3187-f79f-423c-96d2-2b224b6a0b70 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:37.544479 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c3187-f79f-423c-96d2-2b224b6a0b70 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:37.545342 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 21/38] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:16:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:16:37.546705 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c3187-f79f-423c-96d2-2b224b6a0b70 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:37.580674 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2520ac82-a7b7-4dc1-87e9-86d1e6ca2eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:37.581367 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520ac82-a7b7-4dc1-87e9-86d1e6ca2eda None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:37.581653 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520ac82-a7b7-4dc1-87e9-86d1e6ca2eda None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:37.581932 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520ac82-a7b7-4dc1-87e9-86d1e6ca2eda None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:37.582740 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2520ac82-a7b7-4dc1-87e9-86d1e6ca2eda None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:37.584299 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2520ac82-a7b7-4dc1-87e9-86d1e6ca2eda None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:37.602665 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 18/39] 10.222.0.65 () {60 vars in 1131 bytes} [Thu May 19 21:16:37 2022] POST /identity/v3/roles => generated 225 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:16:37.604311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ebdda-2771-4c15-b11a-2d6d04236c62 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:37.604768 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ebdda-2771-4c15-b11a-2d6d04236c62 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:37.605200 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ebdda-2771-4c15-b11a-2d6d04236c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:37.628779 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-600ebdda-2771-4c15-b11a-2d6d04236c62 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:16:37.630051 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-600ebdda-2771-4c15-b11a-2d6d04236c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:16:37.668302 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c3187-f79f-423c-96d2-2b224b6a0b70 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:37.708021 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:37.715586 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 19/40] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:16:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:16:37.715988 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1ab32-5b56-45e9-802b-83394fa99e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:37.716917 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1ab32-5b56-45e9-802b-83394fa99e24 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:37.717282 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1ab32-5b56-45e9-802b-83394fa99e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:37.717618 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1ab32-5b56-45e9-802b-83394fa99e24 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:37.718019 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a5959-83a1-4823-8e21-627c4525f79c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:37.718486 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a5959-83a1-4823-8e21-627c4525f79c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:37.718767 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a5959-83a1-4823-8e21-627c4525f79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:37.727514 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a1ab32-5b56-45e9-802b-83394fa99e24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:37.729492 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a1ab32-5b56-45e9-802b-83394fa99e24 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:37.731629 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 22/41] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:16:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:37.737645 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1ab32-5b56-45e9-802b-83394fa99e24 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:37.750372 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-be7a5959-83a1-4823-8e21-627c4525f79c None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:16:37.751925 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-be7a5959-83a1-4823-8e21-627c4525f79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:16:37.806931 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:37.866071 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 20/42] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:16:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:16:37.867147 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a5959-83a1-4823-8e21-627c4525f79c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:37.889384 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1ab32-5b56-45e9-802b-83394fa99e24 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:37.934944 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9cde4-672e-4bb0-a5ff-e4d5f8281876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:37.935898 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9cde4-672e-4bb0-a5ff-e4d5f8281876 None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:37.936191 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9cde4-672e-4bb0-a5ff-e4d5f8281876 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:37.936482 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9cde4-672e-4bb0-a5ff-e4d5f8281876 None admin] PATH_INFO: `/v3/domains/default/users/d45bbb3805754099870efea6f5d53dfe/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:37.938684 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9cde4-672e-4bb0-a5ff-e4d5f8281876 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d45bbb3805754099870efea6f5d53dfe, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:37.940222 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9cde4-672e-4bb0-a5ff-e4d5f8281876 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:37.960963 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 23/43] 10.222.0.65 () {58 vars in 1361 bytes} [Thu May 19 21:16:37 2022] PUT /identity/v3/domains/default/users/d45bbb3805754099870efea6f5d53dfe/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:16:37.962363 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9cde4-672e-4bb0-a5ff-e4d5f8281876 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:38.017469 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a5959-83a1-4823-8e21-627c4525f79c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:38.052653 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:38.060107 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa213f5-c5d8-43a9-a327-06ec49b3285f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:38.060877 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa213f5-c5d8-43a9-a327-06ec49b3285f None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:38.061222 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa213f5-c5d8-43a9-a327-06ec49b3285f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:38.061506 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa213f5-c5d8-43a9-a327-06ec49b3285f None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:38.062328 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa213f5-c5d8-43a9-a327-06ec49b3285f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:38.063851 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa213f5-c5d8-43a9-a327-06ec49b3285f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:38.079617 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 21/44] 10.222.0.65 () {60 vars in 1131 bytes} [Thu May 19 21:16:37 2022] POST /identity/v3/roles => generated 223 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:16:38.145607 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9cde4-672e-4bb0-a5ff-e4d5f8281876 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:38.174901 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:38.212888 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec83ae-2f38-4566-b8dd-bae718daa13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:38.214374 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec83ae-2f38-4566-b8dd-bae718daa13b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:38.214529 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec83ae-2f38-4566-b8dd-bae718daa13b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:38.214821 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec83ae-2f38-4566-b8dd-bae718daa13b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:38.216237 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adec83ae-2f38-4566-b8dd-bae718daa13b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:38.217772 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adec83ae-2f38-4566-b8dd-bae718daa13b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:38.219598 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 24/45] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:16:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:38.225601 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa213f5-c5d8-43a9-a327-06ec49b3285f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:38.403576 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa213f5-c5d8-43a9-a327-06ec49b3285f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:38.446555 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496ffd2-3bdb-452b-97b4-886d50233d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:38.447238 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496ffd2-3bdb-452b-97b4-886d50233d96 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:38.447570 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496ffd2-3bdb-452b-97b4-886d50233d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:38.447857 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496ffd2-3bdb-452b-97b4-886d50233d96 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:38.448663 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b496ffd2-3bdb-452b-97b4-886d50233d96 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:38.450066 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b496ffd2-3bdb-452b-97b4-886d50233d96 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:38.479281 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 22/46] 10.222.0.65 () {60 vars in 1141 bytes} [Thu May 19 21:16:38 2022] POST /identity/v3/projects => generated 312 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:16:40.367104 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4214f8-c1c7-41e7-86a1-c5b128792c79 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:40.367619 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4214f8-c1c7-41e7-86a1-c5b128792c79 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:40.367701 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4214f8-c1c7-41e7-86a1-c5b128792c79 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:40.369073 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 25/47] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:16:40.375497 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e1beb-1be1-4a45-ae7c-f5aa1b2d8c20 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:40.375909 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e1beb-1be1-4a45-ae7c-f5aa1b2d8c20 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:40.376089 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e1beb-1be1-4a45-ae7c-f5aa1b2d8c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:40.393683 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-205e1beb-1be1-4a45-ae7c-f5aa1b2d8c20 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:16:40.394930 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-205e1beb-1be1-4a45-ae7c-f5aa1b2d8c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:16:40.431759 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:40.467235 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 23/48] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:16:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:16:40.490684 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4214f8-c1c7-41e7-86a1-c5b128792c79 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:40.605704 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4214f8-c1c7-41e7-86a1-c5b128792c79 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:40.641938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5457f-8e32-4f61-9d7a-168685ff2971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:40.642699 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5457f-8e32-4f61-9d7a-168685ff2971 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:40.642987 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5457f-8e32-4f61-9d7a-168685ff2971 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:40.643268 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5457f-8e32-4f61-9d7a-168685ff2971 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:40.654825 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b5457f-8e32-4f61-9d7a-168685ff2971 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:16:40.662573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-c8b5457f-8e32-4f61-9d7a-168685ff2971 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:16:40.662573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:16:40.662573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:16:40.662573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:16:40.662573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:16:40.662573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:16:40.662573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:16:40.662573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:16:40.662573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:16:40.662573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:16:40.662573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:16:40.662573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:16:40.662573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:16:40.662573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:16:40.662573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:16:40.662573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:16:40.663486 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:16:40.663486 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:16:40.663486 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:16:40.663486 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:16:40.663486 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:16:40.663486 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:16:40.663486 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:16:40.663486 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:16:40.663486 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:16:40.663486 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:16:40.663486 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:16:40.663486 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:16:40.663486 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:16:40.663486 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:16:40.663486 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:16:40.663486 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:16:40.664401 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:16:40.664401 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:16:40.664401 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:16:40.664401 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:16:40.664401 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:16:40.664401 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:16:40.664401 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:16:40.664401 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:16:40.664401 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:16:40.664401 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:16:40.664401 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:16:40.664401 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:16:40.664401 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 26/49] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 21:16:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:16:40.670360 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e1beb-1be1-4a45-ae7c-f5aa1b2d8c20 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:40.742930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0634ee-3001-4bd5-aac3-9e68cd21c4a5 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:40.743191 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0634ee-3001-4bd5-aac3-9e68cd21c4a5 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:40.743468 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0634ee-3001-4bd5-aac3-9e68cd21c4a5 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:40.745782 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 27/50] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:16:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:16:40.751304 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a142473-a56a-470a-83e0-10aeb1266305 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:40.751592 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a142473-a56a-470a-83e0-10aeb1266305 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:40.751870 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a142473-a56a-470a-83e0-10aeb1266305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:40.769638 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-1a142473-a56a-470a-83e0-10aeb1266305 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:16:40.771208 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1a142473-a56a-470a-83e0-10aeb1266305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:16:40.778918 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e1beb-1be1-4a45-ae7c-f5aa1b2d8c20 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:40.806200 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:40.815302 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82dbce3-48da-4653-8d08-d53117a77786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:40.815993 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82dbce3-48da-4653-8d08-d53117a77786 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:40.816283 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82dbce3-48da-4653-8d08-d53117a77786 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:40.816566 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82dbce3-48da-4653-8d08-d53117a77786 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:40.817336 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82dbce3-48da-4653-8d08-d53117a77786 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:16:40.817650 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82dbce3-48da-4653-8d08-d53117a77786 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:40.819087 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82dbce3-48da-4653-8d08-d53117a77786 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:40.831222 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 24/51] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:16:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:40.836976 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82dbce3-48da-4653-8d08-d53117a77786 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:40.841464 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 28/52] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:16:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:16:40.864678 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a142473-a56a-470a-83e0-10aeb1266305 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:40.876662 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:40.946800 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82dbce3-48da-4653-8d08-d53117a77786 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:40.968531 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a142473-a56a-470a-83e0-10aeb1266305 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:40.983924 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b24e84-fb88-481a-89ca-4f36c700be95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:40.984810 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b24e84-fb88-481a-89ca-4f36c700be95 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:40.985093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b24e84-fb88-481a-89ca-4f36c700be95 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:40.985374 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b24e84-fb88-481a-89ca-4f36c700be95 None admin] PATH_INFO: `/v3/users/d45bbb3805754099870efea6f5d53dfe` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:40.986970 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b24e84-fb88-481a-89ca-4f36c700be95 None admin] RBAC: Authorizing `identity:get_user(user_id=d45bbb3805754099870efea6f5d53dfe)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:40.988485 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b24e84-fb88-481a-89ca-4f36c700be95 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:40.990187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 25/53] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:16:40 2022] GET /identity/v3/users/d45bbb3805754099870efea6f5d53dfe => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:40.996693 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b24e84-fb88-481a-89ca-4f36c700be95 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:41.011669 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01aaa70-d4bc-4f28-9487-b93d2dd7dc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:41.012489 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01aaa70-d4bc-4f28-9487-b93d2dd7dc67 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:41.012782 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01aaa70-d4bc-4f28-9487-b93d2dd7dc67 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:41.013062 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01aaa70-d4bc-4f28-9487-b93d2dd7dc67 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:41.014323 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01aaa70-d4bc-4f28-9487-b93d2dd7dc67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:41.015783 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01aaa70-d4bc-4f28-9487-b93d2dd7dc67 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:41.021927 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 29/54] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:16:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:16:41.023105 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01aaa70-d4bc-4f28-9487-b93d2dd7dc67 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:41.082683 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:41.133478 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b24e84-fb88-481a-89ca-4f36c700be95 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:41.135417 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:41.156027 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01aaa70-d4bc-4f28-9487-b93d2dd7dc67 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:41.172176 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177de9b6-85d1-473f-982b-a70ed09ce87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:41.172988 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177de9b6-85d1-473f-982b-a70ed09ce87e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:41.173280 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177de9b6-85d1-473f-982b-a70ed09ce87e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:41.173561 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177de9b6-85d1-473f-982b-a70ed09ce87e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:41.174794 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177de9b6-85d1-473f-982b-a70ed09ce87e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:41.176277 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177de9b6-85d1-473f-982b-a70ed09ce87e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:41.178016 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 26/55] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:16:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:41.183901 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177de9b6-85d1-473f-982b-a70ed09ce87e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:41.191194 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089def05-4c4b-4987-a6fd-6011152a9975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:41.191868 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089def05-4c4b-4987-a6fd-6011152a9975 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:41.192148 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089def05-4c4b-4987-a6fd-6011152a9975 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:41.192423 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089def05-4c4b-4987-a6fd-6011152a9975 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:41.193136 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089def05-4c4b-4987-a6fd-6011152a9975 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:41.194636 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089def05-4c4b-4987-a6fd-6011152a9975 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:41.222803 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 30/56] 10.222.0.65 () {60 vars in 1140 bytes} [Thu May 19 21:16:41 2022] POST /identity/v3/projects => generated 298 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:16:41.299930 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177de9b6-85d1-473f-982b-a70ed09ce87e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:41.337982 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f328934e-3baf-4263-9d3f-3b4b11ce359b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:41.338741 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328934e-3baf-4263-9d3f-3b4b11ce359b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:41.339027 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328934e-3baf-4263-9d3f-3b4b11ce359b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:41.339306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328934e-3baf-4263-9d3f-3b4b11ce359b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:41.340181 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f328934e-3baf-4263-9d3f-3b4b11ce359b None admin] RBAC: Adding query filter params (role.id=2b714f2e5c6c469f8fa037bda1ed16f6, scope.domain.id=default, user.id=d45bbb3805754099870efea6f5d53dfe) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:16:41.340489 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f328934e-3baf-4263-9d3f-3b4b11ce359b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:41.341985 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f328934e-3baf-4263-9d3f-3b4b11ce359b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:41.351897 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 27/57] 10.222.0.65 () {56 vars in 1322 bytes} [Thu May 19 21:16:41 2022] GET /identity/v3/role_assignments?user.id=d45bbb3805754099870efea6f5d53dfe&scope.domain.id=default&role.id=2b714f2e5c6c469f8fa037bda1ed16f6 => generated 530 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:16:43.466118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a1229-31e0-4b34-8564-82bee28732ac None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:43.466689 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a1229-31e0-4b34-8564-82bee28732ac None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:43.466689 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a1229-31e0-4b34-8564-82bee28732ac None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:43.468076 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 31/58] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:16:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:16:43.473685 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b6dd9a-e7e1-4c93-862e-a8f8b1d5641c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:43.474073 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b6dd9a-e7e1-4c93-862e-a8f8b1d5641c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:43.474276 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b6dd9a-e7e1-4c93-862e-a8f8b1d5641c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:43.493135 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c1b6dd9a-e7e1-4c93-862e-a8f8b1d5641c None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:16:43.494905 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c1b6dd9a-e7e1-4c93-862e-a8f8b1d5641c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:16:43.522677 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:43.569511 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 28/59] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:16:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:16:43.594179 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a1229-31e0-4b34-8564-82bee28732ac None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:43.694926 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:43.707730 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a1229-31e0-4b34-8564-82bee28732ac None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:43.742615 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91b854-448a-4bc3-b96e-94727811e839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:43.743344 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91b854-448a-4bc3-b96e-94727811e839 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:43.743629 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91b854-448a-4bc3-b96e-94727811e839 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:43.743908 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91b854-448a-4bc3-b96e-94727811e839 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:43.745731 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b91b854-448a-4bc3-b96e-94727811e839 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:43.747216 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b91b854-448a-4bc3-b96e-94727811e839 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:43.748862 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 32/60] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:16:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:43.754750 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b6dd9a-e7e1-4c93-862e-a8f8b1d5641c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:43.860131 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b6dd9a-e7e1-4c93-862e-a8f8b1d5641c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:43.896112 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df7823-3280-414a-98bf-76be44e704ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:43.896779 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df7823-3280-414a-98bf-76be44e704ac None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:43.897069 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df7823-3280-414a-98bf-76be44e704ac None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:43.897346 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df7823-3280-414a-98bf-76be44e704ac None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:43.898059 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6df7823-3280-414a-98bf-76be44e704ac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:43.899570 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6df7823-3280-414a-98bf-76be44e704ac None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:43.926069 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 29/61] 10.222.0.65 () {60 vars in 1132 bytes} [Thu May 19 21:16:43 2022] POST /identity/v3/users => generated 275 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:16:47.434765 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473661d6-d030-4d6e-9b84-59345e8042b0 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:47.435828 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473661d6-d030-4d6e-9b84-59345e8042b0 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:47.436455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473661d6-d030-4d6e-9b84-59345e8042b0 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:47.438049 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 33/62] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:16:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:16:47.439630 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a1b23-1643-4dc6-a30e-301d57a16c6c None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:47.440026 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a1b23-1643-4dc6-a30e-301d57a16c6c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:47.440185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a1b23-1643-4dc6-a30e-301d57a16c6c None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:47.441695 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 30/63] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:16:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:16:47.447491 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e1b49-ed80-4dce-b920-15dccc0b07d1 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:47.447862 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e1b49-ed80-4dce-b920-15dccc0b07d1 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:47.448151 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e1b49-ed80-4dce-b920-15dccc0b07d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:47.450944 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07fd3e2-d8e4-4c08-8da5-9b8510e7d69a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:47.451239 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07fd3e2-d8e4-4c08-8da5-9b8510e7d69a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:47.451525 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07fd3e2-d8e4-4c08-8da5-9b8510e7d69a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:47.468343 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-703e1b49-ed80-4dce-b920-15dccc0b07d1 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:16:47.469904 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-703e1b49-ed80-4dce-b920-15dccc0b07d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:16:47.487145 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a07fd3e2-d8e4-4c08-8da5-9b8510e7d69a None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:16:47.488385 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a07fd3e2-d8e4-4c08-8da5-9b8510e7d69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:16:47.534648 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:47.593360 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 31/64] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:16:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:16:47.603811 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe1f47-217e-4361-b7e9-f12102795e48 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:47.604626 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe1f47-217e-4361-b7e9-f12102795e48 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:47.605939 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe1f47-217e-4361-b7e9-f12102795e48 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:47.608290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 32/65] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:16:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:16:47.612352 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 34/66] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:16:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:16:47.624531 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a689a8-d8d4-470b-9855-48a9c5b23be3 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:47.625044 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a689a8-d8d4-470b-9855-48a9c5b23be3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:47.625459 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a689a8-d8d4-470b-9855-48a9c5b23be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:47.634541 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e1b49-ed80-4dce-b920-15dccc0b07d1 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:47.648786 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-94a689a8-d8d4-470b-9855-48a9c5b23be3 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:16:47.650373 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-94a689a8-d8d4-470b-9855-48a9c5b23be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:16:47.741744 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 33/67] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:16:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:16:47.745637 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:47.747120 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a689a8-d8d4-470b-9855-48a9c5b23be3 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:47.762809 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e1b49-ed80-4dce-b920-15dccc0b07d1 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:47.802684 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce6dac0-5e94-4de9-a01b-31d8897bd42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:47.803501 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce6dac0-5e94-4de9-a01b-31d8897bd42d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:47.803937 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce6dac0-5e94-4de9-a01b-31d8897bd42d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:47.804374 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce6dac0-5e94-4de9-a01b-31d8897bd42d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:47.816115 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce6dac0-5e94-4de9-a01b-31d8897bd42d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:16:47.825281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-cce6dac0-5e94-4de9-a01b-31d8897bd42d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 21:16:47.825281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:16:47.825281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:16:47.825281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:16:47.825281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:16:47.825281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:16:47.825281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:16:47.825281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:16:47.825281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:16:47.825281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:16:47.825281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:16:47.825281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:16:47.825281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:16:47.825281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:16:47.825281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:16:47.825281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:16:47.826346 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:16:47.826346 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:16:47.826346 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:16:47.826346 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:16:47.826346 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:16:47.826346 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:16:47.826346 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:16:47.826346 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:16:47.826346 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:16:47.826346 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:16:47.826346 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:16:47.826346 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:16:47.826346 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:16:47.826346 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:16:47.826346 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:16:47.826346 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:16:47.827281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:16:47.827281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:16:47.827281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:16:47.827281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:16:47.827281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:16:47.827281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:16:47.827281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:16:47.827281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:16:47.827281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:16:47.827281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:16:47.827281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 21:16:47.827281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:16:47.834189 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 35/68] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:16:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:16:47.836007 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c57dbc-cdda-4c6b-a688-039691f470b9 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:47.836436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c57dbc-cdda-4c6b-a688-039691f470b9 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:47.836893 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c57dbc-cdda-4c6b-a688-039691f470b9 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:47.838865 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 36/69] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:16:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:16:47.840077 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c57dbc-cdda-4c6b-a688-039691f470b9 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:47.857903 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:47.879324 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a689a8-d8d4-470b-9855-48a9c5b23be3 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:47.917515 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f85288-3062-4e2f-b41b-869324c2fd32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:47.918943 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f85288-3062-4e2f-b41b-869324c2fd32 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:47.918943 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f85288-3062-4e2f-b41b-869324c2fd32 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:47.918943 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f85288-3062-4e2f-b41b-869324c2fd32 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:47.931417 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f85288-3062-4e2f-b41b-869324c2fd32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:16:47.940750 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-71f85288-3062-4e2f-b41b-869324c2fd32 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:16:47.940750 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:16:47.940750 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:16:47.940750 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:16:47.940750 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:16:47.940750 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:16:47.940750 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:16:47.940750 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:16:47.940750 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:16:47.940750 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:16:47.940750 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:16:47.940750 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:16:47.940750 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:16:47.940750 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:16:47.940750 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:16:47.940750 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:16:47.941877 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:16:47.941877 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:16:47.941877 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:16:47.941877 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:16:47.941877 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:16:47.941877 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:16:47.941877 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:16:47.941877 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:16:47.941877 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:16:47.941877 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:16:47.941877 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:16:47.941877 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:16:47.941877 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:16:47.941877 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:16:47.941877 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:16:47.941877 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:16:47.942701 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:16:47.942701 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:16:47.942701 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:16:47.942701 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:16:47.942701 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:16:47.942701 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:16:47.942701 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:16:47.942701 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:16:47.942701 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:16:47.942701 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:16:47.942701 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:16:47.942701 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:16:47.942701 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 34/70] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 21:16:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:16:47.943519 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f85288-3062-4e2f-b41b-869324c2fd32 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:47.968312 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c57dbc-cdda-4c6b-a688-039691f470b9 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:48.007936 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06473fa6-1033-43bf-8100-5b9780a594f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:48.008684 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06473fa6-1033-43bf-8100-5b9780a594f0 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:48.008967 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06473fa6-1033-43bf-8100-5b9780a594f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:48.009243 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06473fa6-1033-43bf-8100-5b9780a594f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:48.020881 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06473fa6-1033-43bf-8100-5b9780a594f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:16:48.029254 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-06473fa6-1033-43bf-8100-5b9780a594f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:16:48.029254 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:16:48.029254 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:16:48.029254 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:16:48.029254 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:16:48.029254 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:16:48.029254 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:16:48.029254 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:16:48.029254 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:16:48.029254 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:16:48.029254 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:16:48.029254 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:16:48.029254 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:16:48.029254 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:16:48.029254 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:16:48.029254 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:16:48.030154 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:16:48.030154 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:16:48.030154 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:16:48.030154 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:16:48.030154 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:16:48.030154 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:16:48.030154 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:16:48.030154 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:16:48.030154 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:16:48.030154 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:16:48.030154 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:16:48.030154 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:16:48.030154 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:16:48.030154 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:16:48.030154 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:16:48.030154 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:16:48.031109 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:16:48.031109 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:16:48.031109 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:16:48.031109 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:16:48.031109 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:16:48.031109 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:16:48.031109 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:16:48.031109 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:16:48.031109 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:16:48.031109 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:16:48.031109 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:16:48.031109 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:16:48.031109 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 37/71] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 21:16:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:16:48.032188 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f37349-4c43-4723-91a0-0879009ebd54 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:48.032188 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f37349-4c43-4723-91a0-0879009ebd54 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:48.032328 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f37349-4c43-4723-91a0-0879009ebd54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:48.050809 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-a7f37349-4c43-4723-91a0-0879009ebd54 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:16:48.052033 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a7f37349-4c43-4723-91a0-0879009ebd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:16:48.054423 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:48.076014 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f85288-3062-4e2f-b41b-869324c2fd32 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:48.121398 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432798c7-121b-491f-ae9f-865d68fc81c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:48.122071 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432798c7-121b-491f-ae9f-865d68fc81c5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:48.122405 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432798c7-121b-491f-ae9f-865d68fc81c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:48.122693 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432798c7-121b-491f-ae9f-865d68fc81c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:48.123468 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432798c7-121b-491f-ae9f-865d68fc81c5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:16:48.123787 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432798c7-121b-491f-ae9f-865d68fc81c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:48.125241 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432798c7-121b-491f-ae9f-865d68fc81c5 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:48.130252 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 38/72] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:16:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:16:48.132603 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f37349-4c43-4723-91a0-0879009ebd54 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:48.139741 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 35/73] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 21:16:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:48.139741 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432798c7-121b-491f-ae9f-865d68fc81c5 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:48.140683 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:48.237988 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f37349-4c43-4723-91a0-0879009ebd54 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:48.241333 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432798c7-121b-491f-ae9f-865d68fc81c5 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:48.271551 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f23ed0-d8fd-433b-b57c-4286e995a054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:48.272142 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f23ed0-d8fd-433b-b57c-4286e995a054 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:48.272425 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f23ed0-d8fd-433b-b57c-4286e995a054 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:48.272702 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f23ed0-d8fd-433b-b57c-4286e995a054 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:48.273453 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f23ed0-d8fd-433b-b57c-4286e995a054 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:16:48.273760 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f23ed0-d8fd-433b-b57c-4286e995a054 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:48.275409 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f23ed0-d8fd-433b-b57c-4286e995a054 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:48.278298 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf147f4-3a48-4877-888c-bc8928c2acb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:48.278992 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf147f4-3a48-4877-888c-bc8928c2acb3 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:48.279284 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf147f4-3a48-4877-888c-bc8928c2acb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:48.279565 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf147f4-3a48-4877-888c-bc8928c2acb3 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:48.280362 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf147f4-3a48-4877-888c-bc8928c2acb3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:16:48.281102 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf147f4-3a48-4877-888c-bc8928c2acb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:48.281209 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:48.282574 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf147f4-3a48-4877-888c-bc8928c2acb3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:48.285166 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 39/74] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:16:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:48.291161 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0097127-32a1-4c8f-ae46-41027576581b None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:48.292066 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0097127-32a1-4c8f-ae46-41027576581b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:48.292354 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0097127-32a1-4c8f-ae46-41027576581b None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:48.292857 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 36/75] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:16:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:48.293914 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf147f4-3a48-4877-888c-bc8928c2acb3 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:48.294053 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 40/76] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:16:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:16:48.298539 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0097127-32a1-4c8f-ae46-41027576581b None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:48.361378 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:48.401633 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf147f4-3a48-4877-888c-bc8928c2acb3 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:48.407719 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0097127-32a1-4c8f-ae46-41027576581b None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:48.437682 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e67a79-be15-49b8-8052-dd4a2fd4a5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:48.438642 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e67a79-be15-49b8-8052-dd4a2fd4a5ad None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:48.438934 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e67a79-be15-49b8-8052-dd4a2fd4a5ad None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:48.439217 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e67a79-be15-49b8-8052-dd4a2fd4a5ad None admin] PATH_INFO: `/v3/users/b66dd2c6e7694191810a310a769dc893` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:48.442797 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f25b6-d7c2-4904-9838-05f22f3fe3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:48.443612 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f25b6-d7c2-4904-9838-05f22f3fe3f1 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:48.443897 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f25b6-d7c2-4904-9838-05f22f3fe3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:48.444176 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f25b6-d7c2-4904-9838-05f22f3fe3f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:48.455234 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e67a79-be15-49b8-8052-dd4a2fd4a5ad None admin] RBAC: Authorizing `identity:get_user(user_id=b66dd2c6e7694191810a310a769dc893)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:48.455735 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448f25b6-d7c2-4904-9838-05f22f3fe3f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:16:48.456739 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e67a79-be15-49b8-8052-dd4a2fd4a5ad None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:48.458542 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 37/77] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:16:48 2022] GET /identity/v3/users/b66dd2c6e7694191810a310a769dc893 => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:48.459324 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e67a79-be15-49b8-8052-dd4a2fd4a5ad None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:48.464021 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-448f25b6-d7c2-4904-9838-05f22f3fe3f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 21:16:48.464021 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:16:48.464021 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:16:48.464021 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:16:48.464021 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:16:48.464021 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:16:48.464021 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:16:48.464021 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:16:48.464021 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:16:48.464021 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:16:48.464021 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:16:48.464021 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:16:48.464021 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:16:48.464021 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:16:48.464021 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:16:48.464021 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:16:48.465049 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:16:48.465049 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:16:48.465049 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:16:48.465049 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:16:48.465049 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:16:48.465049 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:16:48.465049 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:16:48.465049 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:16:48.465049 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:16:48.465049 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:16:48.465049 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:16:48.465049 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:16:48.465049 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:16:48.465049 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:16:48.465049 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:16:48.465049 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:16:48.465797 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:16:48.465797 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:16:48.465797 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:16:48.465797 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:16:48.465797 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:16:48.465797 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:16:48.465797 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:16:48.465797 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:16:48.465797 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:16:48.465797 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:16:48.465797 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 21:16:48.465797 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:16:48.465797 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 41/78] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:16:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:16:48.466763 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f25b6-d7c2-4904-9838-05f22f3fe3f1 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:48.528222 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:48.569701 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e67a79-be15-49b8-8052-dd4a2fd4a5ad None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:48.570893 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f25b6-d7c2-4904-9838-05f22f3fe3f1 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:48.592542 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:48.605976 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcdb9a1-7e01-4071-bdf4-164f0b2fb77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:48.606169 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa38ff-4acd-4bf7-9631-b8898a66c5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:48.606908 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcdb9a1-7e01-4071-bdf4-164f0b2fb77e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:48.607010 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa38ff-4acd-4bf7-9631-b8898a66c5fb None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:48.607192 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcdb9a1-7e01-4071-bdf4-164f0b2fb77e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:48.607283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa38ff-4acd-4bf7-9631-b8898a66c5fb None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:48.607464 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcdb9a1-7e01-4071-bdf4-164f0b2fb77e None admin] PATH_INFO: `/v3/users/b66dd2c6e7694191810a310a769dc893` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:48.607553 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa38ff-4acd-4bf7-9631-b8898a66c5fb None admin] PATH_INFO: `/v3/users/d45bbb3805754099870efea6f5d53dfe` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:48.608685 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daa38ff-4acd-4bf7-9631-b8898a66c5fb None admin] RBAC: Authorizing `identity:get_user(user_id=d45bbb3805754099870efea6f5d53dfe)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:48.608794 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcdb9a1-7e01-4071-bdf4-164f0b2fb77e None admin] RBAC: Authorizing `identity:get_user(user_id=b66dd2c6e7694191810a310a769dc893)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:48.610154 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daa38ff-4acd-4bf7-9631-b8898a66c5fb None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:48.610265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcdb9a1-7e01-4071-bdf4-164f0b2fb77e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:48.611827 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 38/80] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:16:48 2022] GET /identity/v3/users/b66dd2c6e7694191810a310a769dc893 => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:48.611942 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 42/80] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:16:48 2022] GET /identity/v3/users/d45bbb3805754099870efea6f5d53dfe => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:48.612645 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa38ff-4acd-4bf7-9631-b8898a66c5fb None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:48.613010 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d1e04-2d99-4015-a6ff-95152fe36c24 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:48.613292 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d1e04-2d99-4015-a6ff-95152fe36c24 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:48.613574 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d1e04-2d99-4015-a6ff-95152fe36c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:48.635603 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-431d1e04-2d99-4015-a6ff-95152fe36c24 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:16:48.636853 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-431d1e04-2d99-4015-a6ff-95152fe36c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:16:48.710763 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 39/81] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:16:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:16:48.711637 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d1e04-2d99-4015-a6ff-95152fe36c24 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:48.719054 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa38ff-4acd-4bf7-9631-b8898a66c5fb None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:48.757351 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:48.765705 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478c78fb-34bb-4fad-ae1b-82a1d8a84fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:48.766476 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c78fb-34bb-4fad-ae1b-82a1d8a84fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:48.766764 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c78fb-34bb-4fad-ae1b-82a1d8a84fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:48.767048 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c78fb-34bb-4fad-ae1b-82a1d8a84fc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:48.815837 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d1e04-2d99-4015-a6ff-95152fe36c24 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:48.838080 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:48.838472 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478c78fb-34bb-4fad-ae1b-82a1d8a84fc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:48.840329 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478c78fb-34bb-4fad-ae1b-82a1d8a84fc7 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:48.854009 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36444e4-d358-4028-96ca-00d756d017ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:48.854895 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36444e4-d358-4028-96ca-00d756d017ff None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:48.855245 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36444e4-d358-4028-96ca-00d756d017ff None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:48.855467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36444e4-d358-4028-96ca-00d756d017ff None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:48.856235 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36444e4-d358-4028-96ca-00d756d017ff None admin] RBAC: Adding query filter params (name=member) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:16:48.856544 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36444e4-d358-4028-96ca-00d756d017ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:48.857970 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36444e4-d358-4028-96ca-00d756d017ff None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:48.866827 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 40/82] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 21:16:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:48.867666 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36444e4-d358-4028-96ca-00d756d017ff None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:48.946536 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 43/83] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:16:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:16:48.947507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478c78fb-34bb-4fad-ae1b-82a1d8a84fc7 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:48.972032 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36444e4-d358-4028-96ca-00d756d017ff None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:48.976698 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:49.006847 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a1d3a-d5d3-4dd5-8ce8-3c7197c59ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:49.007510 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a1d3a-d5d3-4dd5-8ce8-3c7197c59ddf None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:49.007796 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a1d3a-d5d3-4dd5-8ce8-3c7197c59ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:49.008078 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a1d3a-d5d3-4dd5-8ce8-3c7197c59ddf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:49.051603 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478c78fb-34bb-4fad-ae1b-82a1d8a84fc7 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:49.061817 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:49.081556 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a1d3a-d5d3-4dd5-8ce8-3c7197c59ddf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:49.083038 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a1d3a-d5d3-4dd5-8ce8-3c7197c59ddf None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:49.088600 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81d99f5-79fe-4ea1-92f5-7902d3945c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:49.089366 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81d99f5-79fe-4ea1-92f5-7902d3945c47 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:49.089654 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81d99f5-79fe-4ea1-92f5-7902d3945c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:49.089934 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81d99f5-79fe-4ea1-92f5-7902d3945c47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:49.162222 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81d99f5-79fe-4ea1-92f5-7902d3945c47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:49.163785 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81d99f5-79fe-4ea1-92f5-7902d3945c47 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:49.184505 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 41/84] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:16:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:16:49.185495 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a1d3a-d5d3-4dd5-8ce8-3c7197c59ddf None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:49.191294 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:49.274725 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 44/85] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:16:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:16:49.274725 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81d99f5-79fe-4ea1-92f5-7902d3945c47 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:49.275077 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:49.293563 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a1d3a-d5d3-4dd5-8ce8-3c7197c59ddf None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:49.330348 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7da618-7abc-43c5-b834-879ffa9678a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:49.331170 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7da618-7abc-43c5-b834-879ffa9678a1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:49.331458 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7da618-7abc-43c5-b834-879ffa9678a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:49.331802 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7da618-7abc-43c5-b834-879ffa9678a1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:49.332969 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7da618-7abc-43c5-b834-879ffa9678a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:49.334513 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7da618-7abc-43c5-b834-879ffa9678a1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:49.336210 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 42/86] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:16:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:49.340851 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7da618-7abc-43c5-b834-879ffa9678a1 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:49.373622 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81d99f5-79fe-4ea1-92f5-7902d3945c47 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:49.401804 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:49.410568 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8f58e-4bfe-4ed5-9c75-c6b20b7990c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:49.411468 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8f58e-4bfe-4ed5-9c75-c6b20b7990c8 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:49.411754 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8f58e-4bfe-4ed5-9c75-c6b20b7990c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:49.412088 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8f58e-4bfe-4ed5-9c75-c6b20b7990c8 None admin] PATH_INFO: `/v3/users/b66dd2c6e7694191810a310a769dc893` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:49.413387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb8f58e-4bfe-4ed5-9c75-c6b20b7990c8 None admin] RBAC: Authorizing `identity:get_user(user_id=b66dd2c6e7694191810a310a769dc893)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:49.414906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb8f58e-4bfe-4ed5-9c75-c6b20b7990c8 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:49.417262 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 45/87] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:16:49 2022] GET /identity/v3/users/b66dd2c6e7694191810a310a769dc893 => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:49.418046 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8f58e-4bfe-4ed5-9c75-c6b20b7990c8 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:49.447200 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7da618-7abc-43c5-b834-879ffa9678a1 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:49.483966 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed70b9a-ec94-4094-b43c-72f35b3c7cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:49.484850 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed70b9a-ec94-4094-b43c-72f35b3c7cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:49.485139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed70b9a-ec94-4094-b43c-72f35b3c7cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:49.485421 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed70b9a-ec94-4094-b43c-72f35b3c7cf9 None admin] PATH_INFO: `/v3/projects/c12a755e369e4c55a7d4a855fc16d986` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:49.486708 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed70b9a-ec94-4094-b43c-72f35b3c7cf9 None admin] RBAC: Authorizing `identity:get_project(project_id=c12a755e369e4c55a7d4a855fc16d986)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:49.488193 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed70b9a-ec94-4094-b43c-72f35b3c7cf9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:49.489993 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 43/88] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:16:49 2022] GET /identity/v3/projects/c12a755e369e4c55a7d4a855fc16d986 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:49.490809 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed70b9a-ec94-4094-b43c-72f35b3c7cf9 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:49.515646 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:49.524579 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8f58e-4bfe-4ed5-9c75-c6b20b7990c8 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:49.561107 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f2a60e-efbe-40d7-b066-d0e1195377b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:49.562008 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2a60e-efbe-40d7-b066-d0e1195377b9 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:49.562332 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2a60e-efbe-40d7-b066-d0e1195377b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:49.562654 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2a60e-efbe-40d7-b066-d0e1195377b9 None admin] PATH_INFO: `/v3/projects/c12a755e369e4c55a7d4a855fc16d986` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:49.563927 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f2a60e-efbe-40d7-b066-d0e1195377b9 None admin] RBAC: Authorizing `identity:get_project(project_id=c12a755e369e4c55a7d4a855fc16d986)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:49.565417 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f2a60e-efbe-40d7-b066-d0e1195377b9 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:49.567265 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 46/89] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:16:49 2022] GET /identity/v3/projects/c12a755e369e4c55a7d4a855fc16d986 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:49.568136 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f2a60e-efbe-40d7-b066-d0e1195377b9 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:49.606192 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed70b9a-ec94-4094-b43c-72f35b3c7cf9 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:49.638064 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:49.642639 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f611dc-0d49-4d97-9710-0946d7b07e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:49.643550 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f611dc-0d49-4d97-9710-0946d7b07e03 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:49.643846 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f611dc-0d49-4d97-9710-0946d7b07e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:49.644126 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f611dc-0d49-4d97-9710-0946d7b07e03 None admin] PATH_INFO: `/v3/projects/c12a755e369e4c55a7d4a855fc16d986` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:49.645355 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f611dc-0d49-4d97-9710-0946d7b07e03 None admin] RBAC: Authorizing `identity:get_project(project_id=c12a755e369e4c55a7d4a855fc16d986)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:49.646875 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f611dc-0d49-4d97-9710-0946d7b07e03 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:49.648806 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 44/90] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:16:49 2022] GET /identity/v3/projects/c12a755e369e4c55a7d4a855fc16d986 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:49.649403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f611dc-0d49-4d97-9710-0946d7b07e03 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:49.679206 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f2a60e-efbe-40d7-b066-d0e1195377b9 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:49.716613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec21b2-e7a3-46eb-89e2-c1d14e88d7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:49.717298 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec21b2-e7a3-46eb-89e2-c1d14e88d7dc None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:49.717583 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec21b2-e7a3-46eb-89e2-c1d14e88d7dc None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:49.717863 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec21b2-e7a3-46eb-89e2-c1d14e88d7dc None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:49.718622 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ec21b2-e7a3-46eb-89e2-c1d14e88d7dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:49.720085 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ec21b2-e7a3-46eb-89e2-c1d14e88d7dc None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:49.740886 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:49.747827 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 47/91] 10.222.0.65 () {60 vars in 1132 bytes} [Thu May 19 21:16:49 2022] POST /identity/v3/users => generated 289 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:16:49.748876 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec21b2-e7a3-46eb-89e2-c1d14e88d7dc None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:49.783156 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f611dc-0d49-4d97-9710-0946d7b07e03 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:49.833021 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a77778e-03b1-43de-9542-f1403a6c3c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:49.833783 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77778e-03b1-43de-9542-f1403a6c3c17 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:49.834140 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77778e-03b1-43de-9542-f1403a6c3c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:49.834647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77778e-03b1-43de-9542-f1403a6c3c17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:49.909460 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec21b2-e7a3-46eb-89e2-c1d14e88d7dc None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:49.932986 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a77778e-03b1-43de-9542-f1403a6c3c17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:49.933774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:49.934548 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a77778e-03b1-43de-9542-f1403a6c3c17 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:49.953862 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da67bc70-bc2a-4b34-aeb9-fb490a1f6617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:49.954570 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da67bc70-bc2a-4b34-aeb9-fb490a1f6617 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:49.954881 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da67bc70-bc2a-4b34-aeb9-fb490a1f6617 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:49.955129 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da67bc70-bc2a-4b34-aeb9-fb490a1f6617 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:49.955991 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da67bc70-bc2a-4b34-aeb9-fb490a1f6617 None admin] RBAC: Adding query filter params (role.id=70699fb0eae7441999835fa3b50350b3, scope.project.id=c12a755e369e4c55a7d4a855fc16d986, user.id=b66dd2c6e7694191810a310a769dc893) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:16:49.956303 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da67bc70-bc2a-4b34-aeb9-fb490a1f6617 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:49.957813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da67bc70-bc2a-4b34-aeb9-fb490a1f6617 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:49.965376 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 48/92] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:16:49 2022] GET /identity/v3/role_assignments?user.id=b66dd2c6e7694191810a310a769dc893&scope.project.id=c12a755e369e4c55a7d4a855fc16d986&role.id=70699fb0eae7441999835fa3b50350b3 => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:49.966298 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da67bc70-bc2a-4b34-aeb9-fb490a1f6617 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:50.070852 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:50.088241 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 45/93] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:16:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:16:50.089056 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a77778e-03b1-43de-9542-f1403a6c3c17 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:50.127457 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da67bc70-bc2a-4b34-aeb9-fb490a1f6617 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:50.171536 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81487a88-56b3-4374-b18c-9965cb8dfcd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:50.172589 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81487a88-56b3-4374-b18c-9965cb8dfcd6 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:50.172868 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81487a88-56b3-4374-b18c-9965cb8dfcd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:50.173148 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81487a88-56b3-4374-b18c-9965cb8dfcd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:50.173998 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81487a88-56b3-4374-b18c-9965cb8dfcd6 None admin] RBAC: Adding query filter params (role.id=2b714f2e5c6c469f8fa037bda1ed16f6, scope.project.id=c12a755e369e4c55a7d4a855fc16d986, user.id=d45bbb3805754099870efea6f5d53dfe) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:16:50.175107 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81487a88-56b3-4374-b18c-9965cb8dfcd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:50.176672 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81487a88-56b3-4374-b18c-9965cb8dfcd6 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:50.184102 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 49/94] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:16:49 2022] GET /identity/v3/role_assignments?user.id=d45bbb3805754099870efea6f5d53dfe&scope.project.id=c12a755e369e4c55a7d4a855fc16d986&role.id=2b714f2e5c6c469f8fa037bda1ed16f6 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:50.185022 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81487a88-56b3-4374-b18c-9965cb8dfcd6 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:50.212813 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a77778e-03b1-43de-9542-f1403a6c3c17 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:50.263115 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:50.273528 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f5114-7ccd-421a-8328-ef85dc89b61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:50.274216 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f5114-7ccd-421a-8328-ef85dc89b61d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:50.274533 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f5114-7ccd-421a-8328-ef85dc89b61d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:50.274812 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f5114-7ccd-421a-8328-ef85dc89b61d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:50.275863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63f5114-7ccd-421a-8328-ef85dc89b61d None admin] RBAC: Adding query filter params (role.id=566c421659204df185f8b77bf3bb3afc, scope.project.id=c12a755e369e4c55a7d4a855fc16d986, user.id=b66dd2c6e7694191810a310a769dc893) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:16:50.276174 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63f5114-7ccd-421a-8328-ef85dc89b61d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:50.277946 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63f5114-7ccd-421a-8328-ef85dc89b61d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:50.297018 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 46/95] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:16:50 2022] GET /identity/v3/role_assignments?user.id=b66dd2c6e7694191810a310a769dc893&scope.project.id=c12a755e369e4c55a7d4a855fc16d986&role.id=566c421659204df185f8b77bf3bb3afc => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:50.357268 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81487a88-56b3-4374-b18c-9965cb8dfcd6 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:50.415935 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be5f83d-0caf-464b-9683-5907bc045bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:50.416890 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5f83d-0caf-464b-9683-5907bc045bfb None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:50.417171 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5f83d-0caf-464b-9683-5907bc045bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:50.417461 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5f83d-0caf-464b-9683-5907bc045bfb None admin] PATH_INFO: `/v3/projects/da0a63de1c7f4c898f8c9db9d8992d51` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:50.418856 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be5f83d-0caf-464b-9683-5907bc045bfb None admin] RBAC: Authorizing `identity:get_project(project_id=da0a63de1c7f4c898f8c9db9d8992d51)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:50.420362 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be5f83d-0caf-464b-9683-5907bc045bfb None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:50.422653 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 50/96] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:16:50 2022] GET /identity/v3/projects/da0a63de1c7f4c898f8c9db9d8992d51 => generated 312 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:50.428459 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f5114-7ccd-421a-8328-ef85dc89b61d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:50.506203 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:50.582453 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f5114-7ccd-421a-8328-ef85dc89b61d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:50.631309 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a821df38-ef45-416a-b6d7-9a63e5ba0a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:50.632043 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821df38-ef45-416a-b6d7-9a63e5ba0a94 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:50.632338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821df38-ef45-416a-b6d7-9a63e5ba0a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:50.632622 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821df38-ef45-416a-b6d7-9a63e5ba0a94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:50.633536 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a821df38-ef45-416a-b6d7-9a63e5ba0a94 None admin] RBAC: Adding query filter params (role.id=70699fb0eae7441999835fa3b50350b3, scope.project.id=da0a63de1c7f4c898f8c9db9d8992d51, user.id=b66dd2c6e7694191810a310a769dc893) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:16:50.633905 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a821df38-ef45-416a-b6d7-9a63e5ba0a94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:50.636605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a821df38-ef45-416a-b6d7-9a63e5ba0a94 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:50.643023 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 47/97] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:16:50 2022] GET /identity/v3/role_assignments?user.id=b66dd2c6e7694191810a310a769dc893&scope.project.id=da0a63de1c7f4c898f8c9db9d8992d51&role.id=70699fb0eae7441999835fa3b50350b3 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:52.877710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a35f7-8395-4a37-ae58-e4beeec4d1c2 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:52.878303 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a35f7-8395-4a37-ae58-e4beeec4d1c2 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:52.878303 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a35f7-8395-4a37-ae58-e4beeec4d1c2 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:52.879694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 51/98] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:16:52.885500 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053dfd3-1725-42e7-a13a-aa5d6947e245 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:52.886381 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053dfd3-1725-42e7-a13a-aa5d6947e245 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:52.886815 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053dfd3-1725-42e7-a13a-aa5d6947e245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:52.904988 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-2053dfd3-1725-42e7-a13a-aa5d6947e245 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:16:52.906521 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2053dfd3-1725-42e7-a13a-aa5d6947e245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:16:53.005306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 48/99] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:16:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:16:53.040742 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a35f7-8395-4a37-ae58-e4beeec4d1c2 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:53.171225 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a35f7-8395-4a37-ae58-e4beeec4d1c2 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:53.198648 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:53.216401 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7ec775-fd42-4d80-996c-69bd848db7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:53.217220 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ec775-fd42-4d80-996c-69bd848db7f5 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:53.217509 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ec775-fd42-4d80-996c-69bd848db7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:53.217793 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ec775-fd42-4d80-996c-69bd848db7f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:53.222990 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7ec775-fd42-4d80-996c-69bd848db7f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:53.224480 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7ec775-fd42-4d80-996c-69bd848db7f5 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:53.226178 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 52/100] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:16:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:53.231774 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053dfd3-1725-42e7-a13a-aa5d6947e245 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:53.280569 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:53.338930 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053dfd3-1725-42e7-a13a-aa5d6947e245 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:53.373222 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41ffb9-ff19-49e3-9394-e6e540711ec5 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:53.373871 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41ffb9-ff19-49e3-9394-e6e540711ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:53.374412 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41ffb9-ff19-49e3-9394-e6e540711ec5 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:53.376025 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 53/101] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:16:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:16:53.377438 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d34740-6115-42cf-a28b-e97045723bcf None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:53.377939 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d34740-6115-42cf-a28b-e97045723bcf None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:53.378496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d34740-6115-42cf-a28b-e97045723bcf None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:53.380007 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 54/102] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:16:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:16:53.381573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ced6d37-e63a-4da2-9be4-80a2bffa5f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:53.382279 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced6d37-e63a-4da2-9be4-80a2bffa5f8c None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:53.382563 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced6d37-e63a-4da2-9be4-80a2bffa5f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:53.382839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced6d37-e63a-4da2-9be4-80a2bffa5f8c None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:53.383562 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ced6d37-e63a-4da2-9be4-80a2bffa5f8c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:53.385073 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ced6d37-e63a-4da2-9be4-80a2bffa5f8c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:53.393340 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7871b234-cd19-4543-8e69-02ede20d3430 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:53.393906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7871b234-cd19-4543-8e69-02ede20d3430 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:53.394425 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7871b234-cd19-4543-8e69-02ede20d3430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:53.412163 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-7871b234-cd19-4543-8e69-02ede20d3430 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:16:53.413729 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7871b234-cd19-4543-8e69-02ede20d3430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:16:53.434450 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 49/103] 10.222.0.65 () {60 vars in 1141 bytes} [Thu May 19 21:16:53 2022] POST /identity/v3/projects => generated 302 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:16:53.435829 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f4fc9-10c8-45fe-86d0-35ca0c13e125 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:53.436122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f4fc9-10c8-45fe-86d0-35ca0c13e125 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:53.436401 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f4fc9-10c8-45fe-86d0-35ca0c13e125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:53.469797 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-8f9f4fc9-10c8-45fe-86d0-35ca0c13e125 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:16:53.471163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8f9f4fc9-10c8-45fe-86d0-35ca0c13e125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:16:53.514952 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 55/104] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:16:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:16:53.579283 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 50/105] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:16:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:16:53.583287 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7871b234-cd19-4543-8e69-02ede20d3430 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:53.654312 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f4fc9-10c8-45fe-86d0-35ca0c13e125 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:53.718791 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:53.776945 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7871b234-cd19-4543-8e69-02ede20d3430 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:53.815695 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:53.824056 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f4fc9-10c8-45fe-86d0-35ca0c13e125 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:53.899154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0a8579-5cca-4899-b3b4-b1fbf2e5fc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:53.901114 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d61a5-199c-43ee-92c1-5d84ac38d103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:53.902599 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d61a5-199c-43ee-92c1-5d84ac38d103 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:53.903042 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d61a5-199c-43ee-92c1-5d84ac38d103 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:53.903777 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d61a5-199c-43ee-92c1-5d84ac38d103 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:53.905792 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a8579-5cca-4899-b3b4-b1fbf2e5fc43 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:53.906478 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a8579-5cca-4899-b3b4-b1fbf2e5fc43 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:53.908953 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a8579-5cca-4899-b3b4-b1fbf2e5fc43 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:53.935372 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0a8579-5cca-4899-b3b4-b1fbf2e5fc43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:16:53.940785 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d61a5-199c-43ee-92c1-5d84ac38d103 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:16:53.955394 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-3f0a8579-5cca-4899-b3b4-b1fbf2e5fc43 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 21:16:53.955394 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:16:53.955394 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:16:53.955394 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:16:53.955394 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:16:53.955394 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:16:53.955394 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:16:53.955394 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:16:53.955394 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:16:53.955394 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:16:53.955394 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:16:53.955394 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:16:53.955394 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:16:53.955394 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:16:53.955394 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:16:53.955394 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:16:53.956331 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:16:53.956331 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:16:53.956331 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:16:53.956331 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:16:53.956331 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:16:53.956331 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:16:53.956331 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:16:53.956331 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:16:53.956331 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:16:53.956331 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:16:53.956331 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:16:53.956331 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:16:53.956331 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:16:53.956331 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:16:53.956331 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:16:53.956331 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:16:53.957115 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:16:53.957115 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:16:53.957115 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:16:53.957115 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:16:53.957115 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:16:53.957115 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:16:53.957115 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:16:53.957115 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:16:53.957115 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:16:53.957115 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:16:53.957115 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 21:16:53.957115 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:16:53.959353 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 51/106] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:16:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:16:53.959707 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-f30d61a5-199c-43ee-92c1-5d84ac38d103 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:16:53.959707 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:16:53.959707 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:16:53.959707 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:16:53.959707 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:16:53.959707 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:16:53.959707 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:16:53.959707 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:16:53.959707 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:16:53.959707 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:16:53.959707 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:16:53.959707 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:16:53.959707 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:16:53.959707 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:16:53.959707 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:16:53.959707 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:16:53.960748 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:16:53.960748 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:16:53.960748 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:16:53.960748 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:16:53.960748 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:16:53.960748 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:16:53.960748 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:16:53.960748 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:16:53.960748 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:16:53.960748 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:16:53.960748 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:16:53.960748 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:16:53.960748 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:16:53.960748 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:16:53.960748 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:16:53.960748 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:16:53.961542 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:16:53.961542 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:16:53.961542 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:16:53.961542 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:16:53.961542 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:16:53.961542 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:16:53.961542 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:16:53.961542 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:16:53.961542 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:16:53.961542 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:16:53.961542 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:16:53.961542 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:16:53.964529 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 56/107] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 21:16:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:16:53.970498 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0a8579-5cca-4899-b3b4-b1fbf2e5fc43 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:53.974692 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d61a5-199c-43ee-92c1-5d84ac38d103 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:54.099886 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:54.121281 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0a8579-5cca-4899-b3b4-b1fbf2e5fc43 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:54.136594 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d61a5-199c-43ee-92c1-5d84ac38d103 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:54.175126 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48a3e3-0b82-48ef-aa25-5d952fab19cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:54.176630 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48a3e3-0b82-48ef-aa25-5d952fab19cb None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:54.177555 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48a3e3-0b82-48ef-aa25-5d952fab19cb None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:54.179332 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48a3e3-0b82-48ef-aa25-5d952fab19cb None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:54.180306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec48a3e3-0b82-48ef-aa25-5d952fab19cb None admin] RBAC: Adding query filter params (name=member) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:16:54.181153 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc50d63-6cf8-46c9-bcc5-ef5768307916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:54.182083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc50d63-6cf8-46c9-bcc5-ef5768307916 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:54.182598 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec48a3e3-0b82-48ef-aa25-5d952fab19cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:54.184177 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec48a3e3-0b82-48ef-aa25-5d952fab19cb None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:54.184657 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc50d63-6cf8-46c9-bcc5-ef5768307916 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:54.185114 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc50d63-6cf8-46c9-bcc5-ef5768307916 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:54.186046 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc50d63-6cf8-46c9-bcc5-ef5768307916 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:16:54.188205 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc50d63-6cf8-46c9-bcc5-ef5768307916 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:54.189816 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc50d63-6cf8-46c9-bcc5-ef5768307916 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:54.203909 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 57/108] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:16:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:54.205549 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2545470-942b-486a-900f-9d2ce6b6b74f None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:54.206026 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2545470-942b-486a-900f-9d2ce6b6b74f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:54.206512 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2545470-942b-486a-900f-9d2ce6b6b74f None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:54.207152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 52/109] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 21:16:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:54.212853 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 58/110] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:16:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:16:54.213062 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee02bc6-967b-4459-8b7a-2f0992517d89 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:54.213062 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee02bc6-967b-4459-8b7a-2f0992517d89 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:54.213062 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee02bc6-967b-4459-8b7a-2f0992517d89 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:54.213062 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 53/111] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:16:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:16:54.213792 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee02bc6-967b-4459-8b7a-2f0992517d89 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:54.214728 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2545470-942b-486a-900f-9d2ce6b6b74f None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:54.255978 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:54.332902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2545470-942b-486a-900f-9d2ce6b6b74f None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:54.351471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee02bc6-967b-4459-8b7a-2f0992517d89 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:54.399860 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb1ee2-ac85-49a3-99e6-38f3b6e858f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:54.401504 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb1ee2-ac85-49a3-99e6-38f3b6e858f9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:54.401815 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e8adb-eef0-4812-9bb0-bd8296aaed23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:54.402218 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb1ee2-ac85-49a3-99e6-38f3b6e858f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:54.402732 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb1ee2-ac85-49a3-99e6-38f3b6e858f9 None admin] PATH_INFO: `/v3/users/b66dd2c6e7694191810a310a769dc893` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:54.402830 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e8adb-eef0-4812-9bb0-bd8296aaed23 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:54.403083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e8adb-eef0-4812-9bb0-bd8296aaed23 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:54.403363 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e8adb-eef0-4812-9bb0-bd8296aaed23 None admin] PATH_INFO: `/v3/users/d45bbb3805754099870efea6f5d53dfe` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:54.404664 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e8adb-eef0-4812-9bb0-bd8296aaed23 None admin] RBAC: Authorizing `identity:get_user(user_id=d45bbb3805754099870efea6f5d53dfe)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:54.404937 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb1ee2-ac85-49a3-99e6-38f3b6e858f9 None admin] RBAC: Authorizing `identity:get_user(user_id=b66dd2c6e7694191810a310a769dc893)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:54.406153 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e8adb-eef0-4812-9bb0-bd8296aaed23 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:54.406824 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb1ee2-ac85-49a3-99e6-38f3b6e858f9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:54.407974 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 59/112] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:16:54 2022] GET /identity/v3/users/d45bbb3805754099870efea6f5d53dfe => generated 247 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:54.412637 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a3c72-9460-4ba9-a0bf-cd0d1a3f1255 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:54.412637 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a3c72-9460-4ba9-a0bf-cd0d1a3f1255 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:54.412637 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a3c72-9460-4ba9-a0bf-cd0d1a3f1255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:54.413958 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 54/113] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:16:54 2022] GET /identity/v3/users/b66dd2c6e7694191810a310a769dc893 => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:54.415996 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f447e-b413-4d88-8620-4a6795a668fe None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:54.416704 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f447e-b413-4d88-8620-4a6795a668fe None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:54.417157 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f447e-b413-4d88-8620-4a6795a668fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:54.437125 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-8a4a3c72-9460-4ba9-a0bf-cd0d1a3f1255 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:16:54.438380 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8a4a3c72-9460-4ba9-a0bf-cd0d1a3f1255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:16:54.439905 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-7a5f447e-b413-4d88-8620-4a6795a668fe None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:16:54.442387 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7a5f447e-b413-4d88-8620-4a6795a668fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:16:54.459608 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:54.518346 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 60/114] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:16:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:16:54.519389 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a3c72-9460-4ba9-a0bf-cd0d1a3f1255 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:54.532593 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 55/115] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:16:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:16:54.533803 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f447e-b413-4d88-8620-4a6795a668fe None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:54.573802 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:54.645725 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a3c72-9460-4ba9-a0bf-cd0d1a3f1255 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:54.653233 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f447e-b413-4d88-8620-4a6795a668fe None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:54.684195 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843c66e3-5816-48a5-a72e-4fb5705c0b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:54.685380 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c66e3-5816-48a5-a72e-4fb5705c0b59 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:54.685833 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c66e3-5816-48a5-a72e-4fb5705c0b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:54.686136 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c66e3-5816-48a5-a72e-4fb5705c0b59 None admin] PATH_INFO: `/v3/projects/c12a755e369e4c55a7d4a855fc16d986` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:54.687524 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843c66e3-5816-48a5-a72e-4fb5705c0b59 None admin] RBAC: Authorizing `identity:get_project(project_id=c12a755e369e4c55a7d4a855fc16d986)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:54.688982 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843c66e3-5816-48a5-a72e-4fb5705c0b59 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:54.695572 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 61/116] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:16:54 2022] GET /identity/v3/projects/c12a755e369e4c55a7d4a855fc16d986 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:54.695572 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843c66e3-5816-48a5-a72e-4fb5705c0b59 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:54.699557 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf69c167-2a8f-4f45-9be4-c4127ef2f57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:54.700477 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69c167-2a8f-4f45-9be4-c4127ef2f57b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:54.700771 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69c167-2a8f-4f45-9be4-c4127ef2f57b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:54.701056 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69c167-2a8f-4f45-9be4-c4127ef2f57b None admin] PATH_INFO: `/v3/projects/c12a755e369e4c55a7d4a855fc16d986` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:54.702355 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf69c167-2a8f-4f45-9be4-c4127ef2f57b None admin] RBAC: Authorizing `identity:get_project(project_id=c12a755e369e4c55a7d4a855fc16d986)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:54.703841 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf69c167-2a8f-4f45-9be4-c4127ef2f57b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:54.705580 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 56/117] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:16:54 2022] GET /identity/v3/projects/c12a755e369e4c55a7d4a855fc16d986 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:54.706861 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855c2ff-d42d-4b38-8a6f-47b2229856b3 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:54.707153 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855c2ff-d42d-4b38-8a6f-47b2229856b3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:54.707434 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855c2ff-d42d-4b38-8a6f-47b2229856b3 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:54.708853 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 57/118] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:16:54.711825 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c855c2ff-d42d-4b38-8a6f-47b2229856b3 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:54.732746 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:54.840711 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c855c2ff-d42d-4b38-8a6f-47b2229856b3 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:54.847386 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843c66e3-5816-48a5-a72e-4fb5705c0b59 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:54.848930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:54.887462 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82591cd9-197e-48d0-900c-58b5649ae960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:54.888300 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82591cd9-197e-48d0-900c-58b5649ae960 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:54.888729 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82591cd9-197e-48d0-900c-58b5649ae960 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:54.889223 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82591cd9-197e-48d0-900c-58b5649ae960 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:54.892062 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5595f9f-9434-4f91-948c-5ec9af1d2ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:54.892894 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5595f9f-9434-4f91-948c-5ec9af1d2ceb None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:54.893176 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5595f9f-9434-4f91-948c-5ec9af1d2ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:54.893573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5595f9f-9434-4f91-948c-5ec9af1d2ceb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:54.905153 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5595f9f-9434-4f91-948c-5ec9af1d2ceb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:16:54.905791 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82591cd9-197e-48d0-900c-58b5649ae960 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:16:54.912929 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-c5595f9f-9434-4f91-948c-5ec9af1d2ceb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 21:16:54.912929 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:16:54.912929 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:16:54.912929 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:16:54.912929 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:16:54.912929 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:16:54.912929 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:16:54.912929 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:16:54.912929 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:16:54.912929 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:16:54.912929 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:16:54.912929 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:16:54.912929 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:16:54.912929 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:16:54.912929 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:16:54.912929 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:16:54.913767 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:16:54.913767 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:16:54.913767 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:16:54.913767 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:16:54.913767 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:16:54.913767 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:16:54.913767 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:16:54.913767 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:16:54.913767 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:16:54.913767 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:16:54.913767 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:16:54.913767 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:16:54.913767 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:16:54.913767 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:16:54.913767 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:16:54.913767 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:16:54.914977 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:16:54.914977 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:16:54.914977 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:16:54.914977 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:16:54.914977 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:16:54.914977 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:16:54.914977 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:16:54.914977 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:16:54.914977 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:16:54.914977 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:16:54.914977 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 21:16:54.914977 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:16:54.916189 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-82591cd9-197e-48d0-900c-58b5649ae960 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:16:54.916189 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:16:54.916189 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:16:54.916189 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:16:54.916189 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:16:54.916189 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:16:54.916189 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:16:54.916189 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:16:54.916189 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:16:54.916189 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:16:54.916189 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:16:54.916189 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:16:54.916189 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:16:54.916189 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:16:54.916189 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:16:54.916189 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:16:54.917610 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:16:54.917610 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:16:54.917610 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:16:54.917610 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:16:54.917610 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:16:54.917610 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:16:54.917610 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:16:54.917610 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:16:54.917610 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:16:54.917610 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:16:54.917610 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:16:54.917610 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:16:54.917610 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:16:54.917610 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:16:54.917610 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:16:54.917610 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:16:54.921145 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:16:54.921145 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:16:54.921145 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:16:54.921145 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:16:54.921145 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:16:54.921145 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:16:54.921145 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:16:54.921145 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:16:54.921145 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:16:54.921145 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:16:54.921145 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:16:54.921145 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:16:54.922615 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 62/119] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:16:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:16:54.922615 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5595f9f-9434-4f91-948c-5ec9af1d2ceb None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:54.922934 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 58/120] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 21:16:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:16:54.922934 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82591cd9-197e-48d0-900c-58b5649ae960 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:54.997756 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:55.033108 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82591cd9-197e-48d0-900c-58b5649ae960 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:55.045212 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5595f9f-9434-4f91-948c-5ec9af1d2ceb None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:55.073051 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bbcad3-7888-4358-92e4-1f99aea93486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:55.074080 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbcad3-7888-4358-92e4-1f99aea93486 None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:55.074601 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbcad3-7888-4358-92e4-1f99aea93486 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:55.074794 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbcad3-7888-4358-92e4-1f99aea93486 None admin] PATH_INFO: `/v3/projects/c12a755e369e4c55a7d4a855fc16d986/users/b66dd2c6e7694191810a310a769dc893/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:55.076896 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bbcad3-7888-4358-92e4-1f99aea93486 None admin] RBAC: Authorizing `identity:create_grant(project_id=c12a755e369e4c55a7d4a855fc16d986, user_id=b66dd2c6e7694191810a310a769dc893, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:55.078403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bbcad3-7888-4358-92e4-1f99aea93486 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:55.082340 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109499a-54d1-4ef6-b1cd-a369d7a9a101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:55.083248 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109499a-54d1-4ef6-b1cd-a369d7a9a101 None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:55.083529 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109499a-54d1-4ef6-b1cd-a369d7a9a101 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:55.083802 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109499a-54d1-4ef6-b1cd-a369d7a9a101 None admin] PATH_INFO: `/v3/projects/c12a755e369e4c55a7d4a855fc16d986/users/d45bbb3805754099870efea6f5d53dfe/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:55.086080 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109499a-54d1-4ef6-b1cd-a369d7a9a101 None admin] RBAC: Authorizing `identity:create_grant(project_id=c12a755e369e4c55a7d4a855fc16d986, user_id=d45bbb3805754099870efea6f5d53dfe, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:55.087646 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109499a-54d1-4ef6-b1cd-a369d7a9a101 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:55.096861 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 59/121] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 21:16:54 2022] PUT /identity/v3/projects/c12a755e369e4c55a7d4a855fc16d986/users/b66dd2c6e7694191810a310a769dc893/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:16:55.097889 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8aee9-023b-42eb-807b-be802dee9964 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:55.098180 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8aee9-023b-42eb-807b-be802dee9964 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:55.098539 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8aee9-023b-42eb-807b-be802dee9964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:55.119380 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c6c8aee9-023b-42eb-807b-be802dee9964 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:16:55.120679 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c6c8aee9-023b-42eb-807b-be802dee9964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:16:55.122276 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 63/122] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 21:16:54 2022] PUT /identity/v3/projects/c12a755e369e4c55a7d4a855fc16d986/users/d45bbb3805754099870efea6f5d53dfe/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:16:55.127451 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109499a-54d1-4ef6-b1cd-a369d7a9a101 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:55.195666 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:55.270657 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 60/123] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:16:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:16:55.271152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c8aee9-023b-42eb-807b-be802dee9964 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:55.340344 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109499a-54d1-4ef6-b1cd-a369d7a9a101 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:55.383841 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ea0cd-58ac-43a6-88c0-a47fe938d6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:55.385189 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ea0cd-58ac-43a6-88c0-a47fe938d6b1 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:55.385950 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ea0cd-58ac-43a6-88c0-a47fe938d6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:55.386496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ea0cd-58ac-43a6-88c0-a47fe938d6b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:55.387656 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7ea0cd-58ac-43a6-88c0-a47fe938d6b1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:16:55.388197 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7ea0cd-58ac-43a6-88c0-a47fe938d6b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:55.389995 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7ea0cd-58ac-43a6-88c0-a47fe938d6b1 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:55.400795 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 64/124] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 21:16:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:55.402293 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ea0cd-58ac-43a6-88c0-a47fe938d6b1 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:55.427003 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:55.492192 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c8aee9-023b-42eb-807b-be802dee9964 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:55.541788 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:55.553765 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a087ff2-3c1b-4c1b-a107-6907dd3a3c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:55.554810 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a087ff2-3c1b-4c1b-a107-6907dd3a3c59 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:55.555088 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a087ff2-3c1b-4c1b-a107-6907dd3a3c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:55.555719 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a087ff2-3c1b-4c1b-a107-6907dd3a3c59 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:55.556942 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a087ff2-3c1b-4c1b-a107-6907dd3a3c59 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:16:55.557212 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a087ff2-3c1b-4c1b-a107-6907dd3a3c59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:55.558874 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a087ff2-3c1b-4c1b-a107-6907dd3a3c59 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:55.571504 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 61/125] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:16:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:55.573061 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a087ff2-3c1b-4c1b-a107-6907dd3a3c59 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:55.589284 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ea0cd-58ac-43a6-88c0-a47fe938d6b1 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:55.684267 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b84a2e1-0ac7-4a84-b8cf-83a110932109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:55.685042 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84a2e1-0ac7-4a84-b8cf-83a110932109 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:55.685326 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84a2e1-0ac7-4a84-b8cf-83a110932109 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:55.685605 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84a2e1-0ac7-4a84-b8cf-83a110932109 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:55.723694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b84a2e1-0ac7-4a84-b8cf-83a110932109 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:16:55.744554 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-2b84a2e1-0ac7-4a84-b8cf-83a110932109 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 21:16:55.744554 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:16:55.744554 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:16:55.744554 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:16:55.744554 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:16:55.744554 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:16:55.744554 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:16:55.744554 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:16:55.744554 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:16:55.744554 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:16:55.744554 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:16:55.744554 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:16:55.744554 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:16:55.744554 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:16:55.744554 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:16:55.744554 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:16:55.745496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:16:55.745496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:16:55.745496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:16:55.745496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:16:55.745496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:16:55.745496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:16:55.745496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:16:55.745496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:16:55.745496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:16:55.745496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:16:55.745496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:16:55.745496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:16:55.745496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:16:55.745496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:16:55.745496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:16:55.745496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:16:55.746295 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:16:55.746295 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:16:55.746295 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:16:55.746295 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:16:55.746295 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:16:55.746295 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:16:55.746295 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:16:55.746295 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:16:55.746295 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:16:55.746295 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:16:55.746295 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 21:16:55.746295 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:16:55.746295 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 65/126] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:16:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:16:55.747023 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b84a2e1-0ac7-4a84-b8cf-83a110932109 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:55.752245 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a087ff2-3c1b-4c1b-a107-6907dd3a3c59 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:55.781858 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:55.797703 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74052795-27d1-4d51-a765-b9971d33676f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:55.799262 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74052795-27d1-4d51-a765-b9971d33676f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:55.799756 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74052795-27d1-4d51-a765-b9971d33676f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:55.800253 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74052795-27d1-4d51-a765-b9971d33676f None admin] PATH_INFO: `/v3/users/1014aa77618f4aaf82cd5198bc2ff5e0` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:55.820782 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74052795-27d1-4d51-a765-b9971d33676f None admin] RBAC: Authorizing `identity:get_user(user_id=1014aa77618f4aaf82cd5198bc2ff5e0)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:55.823015 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74052795-27d1-4d51-a765-b9971d33676f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:55.825171 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 62/127] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:16:55 2022] GET /identity/v3/users/1014aa77618f4aaf82cd5198bc2ff5e0 => generated 289 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:55.825940 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74052795-27d1-4d51-a765-b9971d33676f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:55.952790 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b84a2e1-0ac7-4a84-b8cf-83a110932109 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:56.013394 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74052795-27d1-4d51-a765-b9971d33676f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:56.029126 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603d9c7-ef9f-4e93-8fac-448ebc8f0541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:56.030027 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603d9c7-ef9f-4e93-8fac-448ebc8f0541 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:56.031619 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603d9c7-ef9f-4e93-8fac-448ebc8f0541 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:56.031619 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603d9c7-ef9f-4e93-8fac-448ebc8f0541 None admin] PATH_INFO: `/v3/users/b66dd2c6e7694191810a310a769dc893` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:56.034351 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b603d9c7-ef9f-4e93-8fac-448ebc8f0541 None admin] RBAC: Authorizing `identity:get_user(user_id=b66dd2c6e7694191810a310a769dc893)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:56.036135 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b603d9c7-ef9f-4e93-8fac-448ebc8f0541 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:56.037925 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 66/128] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:16:55 2022] GET /identity/v3/users/b66dd2c6e7694191810a310a769dc893 => generated 275 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:56.039896 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603d9c7-ef9f-4e93-8fac-448ebc8f0541 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:56.061560 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:56.081139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de420c81-a474-479b-85fa-ca843b6db463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:56.081850 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de420c81-a474-479b-85fa-ca843b6db463 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:56.082145 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de420c81-a474-479b-85fa-ca843b6db463 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:56.082481 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de420c81-a474-479b-85fa-ca843b6db463 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:56.083345 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de420c81-a474-479b-85fa-ca843b6db463 None admin] RBAC: Adding query filter params (name=member) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:16:56.083666 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de420c81-a474-479b-85fa-ca843b6db463 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:56.085302 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de420c81-a474-479b-85fa-ca843b6db463 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:56.097484 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 63/129] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 21:16:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:56.098521 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de420c81-a474-479b-85fa-ca843b6db463 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:56.198830 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:56.301286 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603d9c7-ef9f-4e93-8fac-448ebc8f0541 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:56.319432 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de420c81-a474-479b-85fa-ca843b6db463 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:56.364391 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48135d37-85d7-4bce-9fe7-17361954d8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:56.366805 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48135d37-85d7-4bce-9fe7-17361954d8e1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:56.367098 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48135d37-85d7-4bce-9fe7-17361954d8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:56.367381 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48135d37-85d7-4bce-9fe7-17361954d8e1 None admin] PATH_INFO: `/v3/projects/c12a755e369e4c55a7d4a855fc16d986` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:56.369725 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48135d37-85d7-4bce-9fe7-17361954d8e1 None admin] RBAC: Authorizing `identity:get_project(project_id=c12a755e369e4c55a7d4a855fc16d986)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:56.371224 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48135d37-85d7-4bce-9fe7-17361954d8e1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:56.373014 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 64/130] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:16:56 2022] GET /identity/v3/projects/c12a755e369e4c55a7d4a855fc16d986 => generated 298 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:56.373820 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48135d37-85d7-4bce-9fe7-17361954d8e1 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:56.375732 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b1235e-84d6-4b52-b051-c71d3970d53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:56.376636 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1235e-84d6-4b52-b051-c71d3970d53f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:56.377125 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1235e-84d6-4b52-b051-c71d3970d53f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:56.377586 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1235e-84d6-4b52-b051-c71d3970d53f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:56.455645 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b1235e-84d6-4b52-b051-c71d3970d53f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:56.457583 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b1235e-84d6-4b52-b051-c71d3970d53f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:56.477836 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48135d37-85d7-4bce-9fe7-17361954d8e1 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:56.492141 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:56.506801 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:56.517425 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e39aa-1348-462f-a35e-76b8fdfeaa0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:56.518684 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e39aa-1348-462f-a35e-76b8fdfeaa0a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:56.519199 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e39aa-1348-462f-a35e-76b8fdfeaa0a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:56.519647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e39aa-1348-462f-a35e-76b8fdfeaa0a None admin] PATH_INFO: `/v3/users/b66dd2c6e7694191810a310a769dc893` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:56.521081 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5e39aa-1348-462f-a35e-76b8fdfeaa0a None admin] RBAC: Authorizing `identity:get_user(user_id=b66dd2c6e7694191810a310a769dc893)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:56.522909 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5e39aa-1348-462f-a35e-76b8fdfeaa0a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:56.525059 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 65/131] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:16:56 2022] GET /identity/v3/users/b66dd2c6e7694191810a310a769dc893 => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:56.526915 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2fd00-4dcc-4af6-9d30-e0fc13aadff5 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:56.527423 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2fd00-4dcc-4af6-9d30-e0fc13aadff5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:56.527928 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2fd00-4dcc-4af6-9d30-e0fc13aadff5 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:56.529652 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 66/132] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:16:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:16:56.530775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2fd00-4dcc-4af6-9d30-e0fc13aadff5 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:56.569449 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 67/133] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:16:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:16:56.570229 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b1235e-84d6-4b52-b051-c71d3970d53f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:56.656362 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2fd00-4dcc-4af6-9d30-e0fc13aadff5 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:56.683865 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b1235e-84d6-4b52-b051-c71d3970d53f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:56.696328 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0004d2b1-c2b8-412f-95ec-0705b27be186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:56.697870 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004d2b1-c2b8-412f-95ec-0705b27be186 None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:56.698465 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004d2b1-c2b8-412f-95ec-0705b27be186 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:56.698952 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004d2b1-c2b8-412f-95ec-0705b27be186 None admin] PATH_INFO: `/v3/projects/c12a755e369e4c55a7d4a855fc16d986/users/b66dd2c6e7694191810a310a769dc893/roles/566c421659204df185f8b77bf3bb3afc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:56.701507 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0004d2b1-c2b8-412f-95ec-0705b27be186 None admin] RBAC: Authorizing `identity:create_grant(project_id=c12a755e369e4c55a7d4a855fc16d986, user_id=b66dd2c6e7694191810a310a769dc893, role_id=566c421659204df185f8b77bf3bb3afc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:56.703426 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0004d2b1-c2b8-412f-95ec-0705b27be186 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:56.709389 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:56.741124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 67/134] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 21:16:56 2022] PUT /identity/v3/projects/c12a755e369e4c55a7d4a855fc16d986/users/b66dd2c6e7694191810a310a769dc893/roles/566c421659204df185f8b77bf3bb3afc => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:16:56.745613 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddfec9-1d35-47f5-8f5f-e123f9bda163 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:56.747725 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddfec9-1d35-47f5-8f5f-e123f9bda163 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:56.748256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddfec9-1d35-47f5-8f5f-e123f9bda163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:56.768334 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f7323-78c5-460d-b4a8-1c50081b55b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:56.769775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f7323-78c5-460d-b4a8-1c50081b55b5 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:56.770264 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f7323-78c5-460d-b4a8-1c50081b55b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:56.770763 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f7323-78c5-460d-b4a8-1c50081b55b5 None admin] PATH_INFO: `/v3/projects/da0a63de1c7f4c898f8c9db9d8992d51` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:56.772999 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5f7323-78c5-460d-b4a8-1c50081b55b5 None admin] RBAC: Authorizing `identity:get_project(project_id=da0a63de1c7f4c898f8c9db9d8992d51)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:56.775669 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5f7323-78c5-460d-b4a8-1c50081b55b5 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:56.778922 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 68/135] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:16:56 2022] GET /identity/v3/projects/da0a63de1c7f4c898f8c9db9d8992d51 => generated 312 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:56.780394 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f7323-78c5-460d-b4a8-1c50081b55b5 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:56.795482 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-ddddfec9-1d35-47f5-8f5f-e123f9bda163 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:16:56.796718 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ddddfec9-1d35-47f5-8f5f-e123f9bda163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:16:56.905090 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:57.001473 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f7323-78c5-460d-b4a8-1c50081b55b5 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:57.003673 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 68/136] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:16:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:16:57.004521 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddddfec9-1d35-47f5-8f5f-e123f9bda163 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:57.045687 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa10df70-af35-4ae5-8d94-f05992dc7af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:57.046613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10df70-af35-4ae5-8d94-f05992dc7af7 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:57.046897 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10df70-af35-4ae5-8d94-f05992dc7af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:57.047177 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10df70-af35-4ae5-8d94-f05992dc7af7 None admin] PATH_INFO: `/v3/projects/c12a755e369e4c55a7d4a855fc16d986` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:57.049283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa10df70-af35-4ae5-8d94-f05992dc7af7 None admin] RBAC: Authorizing `identity:get_project(project_id=c12a755e369e4c55a7d4a855fc16d986)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:57.050847 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa10df70-af35-4ae5-8d94-f05992dc7af7 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:57.053124 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 69/137] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:16:56 2022] GET /identity/v3/projects/c12a755e369e4c55a7d4a855fc16d986 => generated 298 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:57.054063 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa10df70-af35-4ae5-8d94-f05992dc7af7 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:57.118313 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:57.123128 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddddfec9-1d35-47f5-8f5f-e123f9bda163 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:57.181327 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa10df70-af35-4ae5-8d94-f05992dc7af7 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:57.185806 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d5f15-95d5-4884-9133-2f5f3fafb7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:57.186812 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d5f15-95d5-4884-9133-2f5f3fafb7f3 None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:57.187102 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d5f15-95d5-4884-9133-2f5f3fafb7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:57.187379 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d5f15-95d5-4884-9133-2f5f3fafb7f3 None admin] PATH_INFO: `/v3/projects/da0a63de1c7f4c898f8c9db9d8992d51/users/b66dd2c6e7694191810a310a769dc893/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:57.193562 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0d5f15-95d5-4884-9133-2f5f3fafb7f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=da0a63de1c7f4c898f8c9db9d8992d51, user_id=b66dd2c6e7694191810a310a769dc893, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:57.195146 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0d5f15-95d5-4884-9133-2f5f3fafb7f3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:57.231949 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 69/138] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 21:16:57 2022] PUT /identity/v3/projects/da0a63de1c7f4c898f8c9db9d8992d51/users/b66dd2c6e7694191810a310a769dc893/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:16:57.234054 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d5f15-95d5-4884-9133-2f5f3fafb7f3 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:57.234669 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d013b4-9c85-45ea-adac-a2ee5786e253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:57.236110 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d013b4-9c85-45ea-adac-a2ee5786e253 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:57.237320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d013b4-9c85-45ea-adac-a2ee5786e253 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:57.237713 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d013b4-9c85-45ea-adac-a2ee5786e253 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:57.244326 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d013b4-9c85-45ea-adac-a2ee5786e253 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:57.250194 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d013b4-9c85-45ea-adac-a2ee5786e253 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:57.253724 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 70/139] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:16:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:57.259649 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d013b4-9c85-45ea-adac-a2ee5786e253 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:57.344100 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:57.525514 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d013b4-9c85-45ea-adac-a2ee5786e253 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:57.543295 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d5f15-95d5-4884-9133-2f5f3fafb7f3 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:57.563653 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:57.638761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b384db9-ec6d-45d0-986f-6834f04f44d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:57.639469 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b384db9-ec6d-45d0-986f-6834f04f44d9 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:57.639749 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b384db9-ec6d-45d0-986f-6834f04f44d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:57.640025 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b384db9-ec6d-45d0-986f-6834f04f44d9 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:57.640778 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b384db9-ec6d-45d0-986f-6834f04f44d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:57.642351 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b384db9-ec6d-45d0-986f-6834f04f44d9 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:57.647951 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c86dab-b314-4671-9b87-c928e341c7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:57.648639 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c86dab-b314-4671-9b87-c928e341c7c6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:57.648930 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c86dab-b314-4671-9b87-c928e341c7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:57.649213 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c86dab-b314-4671-9b87-c928e341c7c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:57.650141 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c86dab-b314-4671-9b87-c928e341c7c6 None admin] RBAC: Adding query filter params (role.id=34a7a66da3654831a115d77e40a989da, scope.project.id=c12a755e369e4c55a7d4a855fc16d986, user.id=1014aa77618f4aaf82cd5198bc2ff5e0) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:16:57.651234 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c86dab-b314-4671-9b87-c928e341c7c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:57.652737 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c86dab-b314-4671-9b87-c928e341c7c6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:57.667921 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 70/140] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:16:57 2022] GET /identity/v3/role_assignments?user.id=1014aa77618f4aaf82cd5198bc2ff5e0&scope.project.id=c12a755e369e4c55a7d4a855fc16d986&role.id=34a7a66da3654831a115d77e40a989da => generated 259 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:57.689123 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 71/141] 10.222.0.65 () {60 vars in 1132 bytes} [Thu May 19 21:16:57 2022] POST /identity/v3/users => generated 283 bytes in 430 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:16:58.160327 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913dc76-2d77-4fe9-b16a-4691db365ac6 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:58.161134 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913dc76-2d77-4fe9-b16a-4691db365ac6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:58.161134 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913dc76-2d77-4fe9-b16a-4691db365ac6 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:58.162528 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 71/142] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:16:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:16:58.169659 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966b66d-de4a-427e-80f8-bc5c28bd1c14 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:58.170474 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966b66d-de4a-427e-80f8-bc5c28bd1c14 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:58.170474 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966b66d-de4a-427e-80f8-bc5c28bd1c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:58.195537 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-3966b66d-de4a-427e-80f8-bc5c28bd1c14 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:16:58.196800 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3966b66d-de4a-427e-80f8-bc5c28bd1c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:16:58.293504 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 72/143] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:16:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:16:58.329195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2913dc76-2d77-4fe9-b16a-4691db365ac6 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:58.447333 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:58.454233 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2913dc76-2d77-4fe9-b16a-4691db365ac6 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:58.502132 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb17180-8369-410d-8f23-913c29287df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:58.502996 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb17180-8369-410d-8f23-913c29287df7 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:58.503279 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb17180-8369-410d-8f23-913c29287df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:58.503561 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb17180-8369-410d-8f23-913c29287df7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:58.520859 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb17180-8369-410d-8f23-913c29287df7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:16:58.528968 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-bdb17180-8369-410d-8f23-913c29287df7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 21:16:58.528968 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:16:58.528968 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:16:58.528968 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:16:58.528968 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:16:58.528968 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:16:58.528968 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:16:58.528968 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:16:58.528968 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:16:58.528968 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:16:58.528968 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:16:58.528968 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:16:58.528968 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:16:58.528968 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:16:58.528968 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:16:58.528968 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:16:58.530856 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:16:58.530856 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:16:58.530856 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:16:58.530856 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:16:58.530856 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:16:58.530856 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:16:58.530856 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:16:58.530856 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:16:58.530856 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:16:58.530856 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:16:58.530856 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:16:58.530856 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:16:58.530856 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:16:58.530856 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:16:58.530856 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:16:58.530856 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:16:58.532376 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:16:58.532376 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:16:58.532376 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:16:58.532376 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:16:58.532376 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:16:58.532376 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:16:58.532376 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:16:58.532376 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:16:58.532376 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:16:58.532376 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:16:58.532376 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 21:16:58.532376 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:16:58.532376 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 72/144] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:16:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:16:58.540060 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3966b66d-de4a-427e-80f8-bc5c28bd1c14 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:58.615344 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17dfdc-f44e-42d1-a881-59ebe491fd68 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:58.615777 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17dfdc-f44e-42d1-a881-59ebe491fd68 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:58.616013 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17dfdc-f44e-42d1-a881-59ebe491fd68 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:58.617290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 73/145] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:16:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:16:58.627135 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c112cb-e693-455a-bbcf-0fa885f5c67a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:58.627574 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c112cb-e693-455a-bbcf-0fa885f5c67a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:58.627808 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c112cb-e693-455a-bbcf-0fa885f5c67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:58.655661 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-87c112cb-e693-455a-bbcf-0fa885f5c67a None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:16:58.656908 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-87c112cb-e693-455a-bbcf-0fa885f5c67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:16:58.706664 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3966b66d-de4a-427e-80f8-bc5c28bd1c14 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:58.709019 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:58.777150 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b956dcf8-e7d8-4105-9219-bcac130fe66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:58.777897 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b956dcf8-e7d8-4105-9219-bcac130fe66b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:58.778180 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b956dcf8-e7d8-4105-9219-bcac130fe66b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:58.778493 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b956dcf8-e7d8-4105-9219-bcac130fe66b None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:58.779263 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b956dcf8-e7d8-4105-9219-bcac130fe66b None admin] RBAC: Adding query filter params (name=member) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:16:58.779569 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b956dcf8-e7d8-4105-9219-bcac130fe66b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:58.780971 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b956dcf8-e7d8-4105-9219-bcac130fe66b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:58.798632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 73/146] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 21:16:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:58.804982 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b956dcf8-e7d8-4105-9219-bcac130fe66b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:58.821825 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 74/147] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:16:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:16:58.920899 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c112cb-e693-455a-bbcf-0fa885f5c67a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:58.936316 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b956dcf8-e7d8-4105-9219-bcac130fe66b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:58.975092 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da717fd4-b5f4-4ceb-83b3-9fd8bc33600f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:58.976481 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da717fd4-b5f4-4ceb-83b3-9fd8bc33600f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:58.977789 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da717fd4-b5f4-4ceb-83b3-9fd8bc33600f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:58.979576 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da717fd4-b5f4-4ceb-83b3-9fd8bc33600f None admin] PATH_INFO: `/v3/users/b66dd2c6e7694191810a310a769dc893` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:58.981170 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da717fd4-b5f4-4ceb-83b3-9fd8bc33600f None admin] RBAC: Authorizing `identity:get_user(user_id=b66dd2c6e7694191810a310a769dc893)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:58.983894 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da717fd4-b5f4-4ceb-83b3-9fd8bc33600f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:58.986028 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 74/148] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:16:58 2022] GET /identity/v3/users/b66dd2c6e7694191810a310a769dc893 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:58.994409 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da717fd4-b5f4-4ceb-83b3-9fd8bc33600f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:59.061104 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:59.083634 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:59.107650 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c112cb-e693-455a-bbcf-0fa885f5c67a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:59.167719 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca58424-ee58-49a8-a204-fbc41b6b9509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:59.169347 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca58424-ee58-49a8-a204-fbc41b6b9509 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:59.169820 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca58424-ee58-49a8-a204-fbc41b6b9509 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:59.170332 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca58424-ee58-49a8-a204-fbc41b6b9509 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:59.192381 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca58424-ee58-49a8-a204-fbc41b6b9509 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:16:59.206155 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-cca58424-ee58-49a8-a204-fbc41b6b9509 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:16:59.206155 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:16:59.206155 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:16:59.206155 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:16:59.206155 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:16:59.206155 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:16:59.206155 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:16:59.206155 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:16:59.206155 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:16:59.206155 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:16:59.206155 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:16:59.206155 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:16:59.206155 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:16:59.206155 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:16:59.206155 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:16:59.206155 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:16:59.207099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:16:59.207099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:16:59.207099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:16:59.207099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:16:59.207099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:16:59.207099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:16:59.207099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:16:59.207099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:16:59.207099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:16:59.207099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:16:59.207099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:16:59.207099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:16:59.207099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:16:59.207099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:16:59.207099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:16:59.207099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:16:59.207851 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:16:59.207851 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:16:59.207851 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:16:59.207851 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:16:59.207851 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:16:59.207851 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:16:59.207851 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:16:59.207851 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:16:59.207851 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:16:59.207851 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:16:59.207851 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:16:59.207851 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:16:59.210359 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 75/149] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 21:16:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:16:59.219629 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca58424-ee58-49a8-a204-fbc41b6b9509 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:59.292234 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da717fd4-b5f4-4ceb-83b3-9fd8bc33600f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:59.363209 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0034f5-3bbe-4c6d-910c-b75f555b9ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:59.363890 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0034f5-3bbe-4c6d-910c-b75f555b9ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:59.364175 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0034f5-3bbe-4c6d-910c-b75f555b9ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:59.364450 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0034f5-3bbe-4c6d-910c-b75f555b9ea4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:59.369358 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca58424-ee58-49a8-a204-fbc41b6b9509 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:59.411256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74beccf7-0954-4a90-8686-f1eea1a7adfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:59.412535 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74beccf7-0954-4a90-8686-f1eea1a7adfe None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:59.413278 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74beccf7-0954-4a90-8686-f1eea1a7adfe None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:59.413754 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74beccf7-0954-4a90-8686-f1eea1a7adfe None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:59.414747 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74beccf7-0954-4a90-8686-f1eea1a7adfe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:16:59.415283 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74beccf7-0954-4a90-8686-f1eea1a7adfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:59.417056 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74beccf7-0954-4a90-8686-f1eea1a7adfe None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:59.427449 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 76/150] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:16:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:59.432754 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74beccf7-0954-4a90-8686-f1eea1a7adfe None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:59.448080 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:59.499422 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0034f5-3bbe-4c6d-910c-b75f555b9ea4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:59.500996 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0034f5-3bbe-4c6d-910c-b75f555b9ea4 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:59.578781 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:59.600820 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74beccf7-0954-4a90-8686-f1eea1a7adfe None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:59.677071 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7a65d-97c0-4fa9-a93d-b5c038558bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:59.678553 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7a65d-97c0-4fa9-a93d-b5c038558bba None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:59.679042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7a65d-97c0-4fa9-a93d-b5c038558bba None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:59.680456 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7a65d-97c0-4fa9-a93d-b5c038558bba None admin] PATH_INFO: `/v3/users/d45bbb3805754099870efea6f5d53dfe` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:59.682759 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c7a65d-97c0-4fa9-a93d-b5c038558bba None admin] RBAC: Authorizing `identity:get_user(user_id=d45bbb3805754099870efea6f5d53dfe)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:16:59.685272 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c7a65d-97c0-4fa9-a93d-b5c038558bba None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:16:59.688037 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 77/151] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:16:59 2022] GET /identity/v3/users/d45bbb3805754099870efea6f5d53dfe => generated 247 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:16:59.693705 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7a65d-97c0-4fa9-a93d-b5c038558bba None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:59.735373 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 75/152] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:16:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 742 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:16:59.741911 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0034f5-3bbe-4c6d-910c-b75f555b9ea4 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:16:59.836796 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:16:59.869325 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7a65d-97c0-4fa9-a93d-b5c038558bba None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:16:59.909371 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32463da-b13a-4826-84b9-f59b53ee91a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:16:59.910434 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32463da-b13a-4826-84b9-f59b53ee91a3 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:16:59.910923 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32463da-b13a-4826-84b9-f59b53ee91a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:16:59.911383 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32463da-b13a-4826-84b9-f59b53ee91a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:16:59.969977 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0034f5-3bbe-4c6d-910c-b75f555b9ea4 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:00.013510 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32463da-b13a-4826-84b9-f59b53ee91a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:00.015688 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32463da-b13a-4826-84b9-f59b53ee91a3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:00.026422 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:00.039542 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1068717-dff2-4bf6-b23d-a890644343ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:00.040448 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1068717-dff2-4bf6-b23d-a890644343ec None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:00.040918 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1068717-dff2-4bf6-b23d-a890644343ec None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:00.041208 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1068717-dff2-4bf6-b23d-a890644343ec None admin] PATH_INFO: `/v3/projects/c12a755e369e4c55a7d4a855fc16d986` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:00.046260 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1068717-dff2-4bf6-b23d-a890644343ec None admin] RBAC: Authorizing `identity:get_project(project_id=c12a755e369e4c55a7d4a855fc16d986)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:00.046260 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1068717-dff2-4bf6-b23d-a890644343ec None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:00.047870 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 76/153] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:16:59 2022] GET /identity/v3/projects/c12a755e369e4c55a7d4a855fc16d986 => generated 298 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:17:00.054829 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1068717-dff2-4bf6-b23d-a890644343ec None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:00.136421 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:00.171041 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 78/154] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:16:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 21:17:00.176819 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32463da-b13a-4826-84b9-f59b53ee91a3 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:00.312508 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1068717-dff2-4bf6-b23d-a890644343ec None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:00.349309 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32463da-b13a-4826-84b9-f59b53ee91a3 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:00.389943 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d749b9ea-7832-41e2-9683-bb82e2e395a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:00.390925 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749b9ea-7832-41e2-9683-bb82e2e395a7 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:00.391350 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749b9ea-7832-41e2-9683-bb82e2e395a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:00.391780 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749b9ea-7832-41e2-9683-bb82e2e395a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:00.392780 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d749b9ea-7832-41e2-9683-bb82e2e395a7 None admin] RBAC: Adding query filter params (role.id=70699fb0eae7441999835fa3b50350b3, scope.project.id=c12a755e369e4c55a7d4a855fc16d986, user.id=b66dd2c6e7694191810a310a769dc893) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:00.393241 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d749b9ea-7832-41e2-9683-bb82e2e395a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:00.394923 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d749b9ea-7832-41e2-9683-bb82e2e395a7 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:00.404756 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 77/155] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:17:00 2022] GET /identity/v3/role_assignments?user.id=b66dd2c6e7694191810a310a769dc893&scope.project.id=c12a755e369e4c55a7d4a855fc16d986&role.id=70699fb0eae7441999835fa3b50350b3 => generated 608 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:00.407849 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511398f9-38d9-48ca-bd9f-eed5c2b58ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:00.408884 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511398f9-38d9-48ca-bd9f-eed5c2b58ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:00.409265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511398f9-38d9-48ca-bd9f-eed5c2b58ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:00.409639 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511398f9-38d9-48ca-bd9f-eed5c2b58ba4 None admin] PATH_INFO: `/v3/projects/c12a755e369e4c55a7d4a855fc16d986` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:00.411070 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511398f9-38d9-48ca-bd9f-eed5c2b58ba4 None admin] RBAC: Authorizing `identity:get_project(project_id=c12a755e369e4c55a7d4a855fc16d986)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:00.412666 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511398f9-38d9-48ca-bd9f-eed5c2b58ba4 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:00.414624 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 79/156] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:17:00 2022] GET /identity/v3/projects/c12a755e369e4c55a7d4a855fc16d986 => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:00.421026 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d749b9ea-7832-41e2-9683-bb82e2e395a7 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:00.491114 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:00.534224 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d749b9ea-7832-41e2-9683-bb82e2e395a7 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:00.575564 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a748b-90d2-4b52-bce0-34b9d2c49329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:00.576247 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a748b-90d2-4b52-bce0-34b9d2c49329 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:00.576526 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a748b-90d2-4b52-bce0-34b9d2c49329 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:00.576800 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a748b-90d2-4b52-bce0-34b9d2c49329 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:00.577653 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6a748b-90d2-4b52-bce0-34b9d2c49329 None admin] RBAC: Adding query filter params (role.id=2b714f2e5c6c469f8fa037bda1ed16f6, scope.project.id=c12a755e369e4c55a7d4a855fc16d986, user.id=d45bbb3805754099870efea6f5d53dfe) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:00.577964 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6a748b-90d2-4b52-bce0-34b9d2c49329 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:00.579459 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6a748b-90d2-4b52-bce0-34b9d2c49329 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:00.586763 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 78/157] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:17:00 2022] GET /identity/v3/role_assignments?user.id=d45bbb3805754099870efea6f5d53dfe&scope.project.id=c12a755e369e4c55a7d4a855fc16d986&role.id=2b714f2e5c6c469f8fa037bda1ed16f6 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:01.116938 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998aa225-793b-42d1-a50a-8582aa57bafe None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:01.117620 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998aa225-793b-42d1-a50a-8582aa57bafe None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:01.118114 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998aa225-793b-42d1-a50a-8582aa57bafe None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:01.120421 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 80/158] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:01.132209 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403410f-2fca-4957-b0d3-91925def0862 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:01.132826 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403410f-2fca-4957-b0d3-91925def0862 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:01.133310 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403410f-2fca-4957-b0d3-91925def0862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:01.155693 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-5403410f-2fca-4957-b0d3-91925def0862 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:01.157230 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5403410f-2fca-4957-b0d3-91925def0862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:01.245179 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 79/159] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:17:01.286640 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998aa225-793b-42d1-a50a-8582aa57bafe None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:01.342014 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:01.472720 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998aa225-793b-42d1-a50a-8582aa57bafe None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:01.538894 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a1b217-d50e-4448-9bff-d8188b5ac686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:01.540320 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1b217-d50e-4448-9bff-d8188b5ac686 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:01.540911 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1b217-d50e-4448-9bff-d8188b5ac686 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:01.541335 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1b217-d50e-4448-9bff-d8188b5ac686 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:01.565536 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a1b217-d50e-4448-9bff-d8188b5ac686 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:01.579326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-e1a1b217-d50e-4448-9bff-d8188b5ac686 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:17:01.579326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:01.579326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:01.579326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:01.579326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:01.579326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:01.579326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:01.579326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:01.579326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:01.579326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:01.579326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:01.579326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:01.579326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:01.579326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:01.579326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:01.579326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:01.580338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:01.580338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:01.580338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:01.580338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:01.580338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:01.580338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:01.580338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:01.580338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:01.580338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:01.580338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:01.580338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:01.580338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:17:01.580338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:01.580338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:01.580338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:01.580338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:01.581106 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:01.581106 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:01.581106 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:01.581106 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:01.581106 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:01.581106 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:01.581106 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:01.581106 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:01.581106 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:01.581106 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:01.581106 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:17:01.581106 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:17:01.581106 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 81/160] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 21:17:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:01.587425 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5403410f-2fca-4957-b0d3-91925def0862 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:01.667311 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:01.677560 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132acf8-d0cd-4cea-bbfd-44f25e74b508 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:01.677901 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132acf8-d0cd-4cea-bbfd-44f25e74b508 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:01.678115 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132acf8-d0cd-4cea-bbfd-44f25e74b508 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:01.679670 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 82/161] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:01.685635 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e1650-f1c3-47b7-9cf4-a5552d2fe2f3 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:01.685899 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e1650-f1c3-47b7-9cf4-a5552d2fe2f3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:01.686172 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e1650-f1c3-47b7-9cf4-a5552d2fe2f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:01.723687 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-1c3e1650-f1c3-47b7-9cf4-a5552d2fe2f3 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:01.724822 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1c3e1650-f1c3-47b7-9cf4-a5552d2fe2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:01.768658 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5403410f-2fca-4957-b0d3-91925def0862 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:01.817248 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 83/162] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 19 21:17:01.818931 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef592c4d-e414-4ed8-8a49-9382226c0254 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:01.819219 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef592c4d-e414-4ed8-8a49-9382226c0254 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:01.819492 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef592c4d-e414-4ed8-8a49-9382226c0254 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:01.821073 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 84/163] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:01.827036 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4063a23-adb2-46fd-9603-8f8db67c89cf None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:01.827319 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4063a23-adb2-46fd-9603-8f8db67c89cf None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:01.827594 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4063a23-adb2-46fd-9603-8f8db67c89cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:01.871390 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe95c995-b8b5-467e-b377-1481ff19c4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:01.872079 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95c995-b8b5-467e-b377-1481ff19c4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:01.872354 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95c995-b8b5-467e-b377-1481ff19c4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:01.872604 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95c995-b8b5-467e-b377-1481ff19c4f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:01.873378 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe95c995-b8b5-467e-b377-1481ff19c4f2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:01.873679 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe95c995-b8b5-467e-b377-1481ff19c4f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:01.875401 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e4063a23-adb2-46fd-9603-8f8db67c89cf None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:01.875672 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe95c995-b8b5-467e-b377-1481ff19c4f2 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:01.876563 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e4063a23-adb2-46fd-9603-8f8db67c89cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:01.886142 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 80/164] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:17:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:01.886816 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe95c995-b8b5-467e-b377-1481ff19c4f2 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:01.948098 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:02.000275 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe95c995-b8b5-467e-b377-1481ff19c4f2 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:02.021321 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:02.055756 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 85/165] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:17:02.057152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4063a23-adb2-46fd-9603-8f8db67c89cf None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:02.058766 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac543019-2d8a-459b-86df-a8d9d00287c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:02.059584 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac543019-2d8a-459b-86df-a8d9d00287c0 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:02.059862 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac543019-2d8a-459b-86df-a8d9d00287c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:02.060134 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac543019-2d8a-459b-86df-a8d9d00287c0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:02.079859 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac543019-2d8a-459b-86df-a8d9d00287c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:02.089401 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-ac543019-2d8a-459b-86df-a8d9d00287c0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 21:17:02.089401 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:02.089401 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:02.089401 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:02.089401 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:02.089401 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:02.089401 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:02.089401 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:02.089401 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:02.089401 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:02.089401 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:02.089401 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:02.089401 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:02.089401 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:02.089401 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:02.089401 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:02.091030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:02.091030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:02.091030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:02.091030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:02.091030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:02.091030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:02.091030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:02.091030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:02.091030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:02.091030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:02.091030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:02.091030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:02.091030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:02.091030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:02.091030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:02.091030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:02.093223 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:02.093223 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:02.093223 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:02.093223 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:02.093223 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:02.093223 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:02.093223 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:02.093223 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:02.093223 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:02.093223 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:02.093223 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 21:17:02.093223 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:02.093223 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 81/166] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:17:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:02.099607 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac543019-2d8a-459b-86df-a8d9d00287c0 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:02.209705 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac543019-2d8a-459b-86df-a8d9d00287c0 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:02.253431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4063a23-adb2-46fd-9603-8f8db67c89cf None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:02.265091 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ed726c-e739-4316-947b-36700e6cc704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:02.265796 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed726c-e739-4316-947b-36700e6cc704 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:02.266083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed726c-e739-4316-947b-36700e6cc704 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:02.266390 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed726c-e739-4316-947b-36700e6cc704 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:02.267169 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ed726c-e739-4316-947b-36700e6cc704 None admin] RBAC: Adding query filter params (name=member) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:02.267490 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ed726c-e739-4316-947b-36700e6cc704 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:02.268923 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ed726c-e739-4316-947b-36700e6cc704 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:02.288025 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 82/167] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 21:17:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:17:02.289314 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ed726c-e739-4316-947b-36700e6cc704 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:02.315803 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e99d55-000a-4813-a29e-63549b460fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:02.320361 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e99d55-000a-4813-a29e-63549b460fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:02.321013 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e99d55-000a-4813-a29e-63549b460fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:02.322509 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e99d55-000a-4813-a29e-63549b460fb9 None admin] PATH_INFO: `/v3/users/b66dd2c6e7694191810a310a769dc893` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:02.328091 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e99d55-000a-4813-a29e-63549b460fb9 None admin] RBAC: Authorizing `identity:get_user(user_id=b66dd2c6e7694191810a310a769dc893)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:02.334904 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e99d55-000a-4813-a29e-63549b460fb9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:02.336921 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 86/168] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:17:02 2022] GET /identity/v3/users/b66dd2c6e7694191810a310a769dc893 => generated 275 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:02.337747 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e99d55-000a-4813-a29e-63549b460fb9 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:02.374292 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:02.424890 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:02.442060 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ed726c-e739-4316-947b-36700e6cc704 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:02.479093 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d23d08-1188-4c43-871a-cba44298e274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:02.480198 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d23d08-1188-4c43-871a-cba44298e274 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:02.480615 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d23d08-1188-4c43-871a-cba44298e274 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:02.481067 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d23d08-1188-4c43-871a-cba44298e274 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:02.492564 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d23d08-1188-4c43-871a-cba44298e274 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:02.501437 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-c6d23d08-1188-4c43-871a-cba44298e274 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 21:17:02.501437 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:02.501437 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:02.501437 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:02.501437 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:02.501437 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:02.501437 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:02.501437 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:02.501437 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:02.501437 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:02.501437 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:02.501437 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:02.501437 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:02.501437 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:02.501437 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:02.501437 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:02.502608 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:02.502608 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:02.502608 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:02.502608 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:02.502608 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:02.502608 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:02.502608 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:02.502608 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:02.502608 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:02.502608 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:02.502608 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:02.502608 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:02.502608 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:02.502608 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:02.502608 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:02.502608 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:02.503635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:02.503635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:02.503635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:02.503635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:02.503635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:02.503635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:02.503635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:02.503635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:02.503635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:02.503635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:02.503635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 21:17:02.503635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:02.505837 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 83/169] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:17:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:02.506841 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d23d08-1188-4c43-871a-cba44298e274 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:02.624277 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d23d08-1188-4c43-871a-cba44298e274 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:02.626652 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:02.628831 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e99d55-000a-4813-a29e-63549b460fb9 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:02.664330 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee0666-f1ed-4373-8e64-eed9c86fbab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:02.665288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee0666-f1ed-4373-8e64-eed9c86fbab5 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:02.665826 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee0666-f1ed-4373-8e64-eed9c86fbab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:02.666313 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee0666-f1ed-4373-8e64-eed9c86fbab5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:02.737136 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f869745-4b6a-47ef-9a7b-8ebb67cf8138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:02.738079 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f869745-4b6a-47ef-9a7b-8ebb67cf8138 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:02.738422 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f869745-4b6a-47ef-9a7b-8ebb67cf8138 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:02.738693 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f869745-4b6a-47ef-9a7b-8ebb67cf8138 None admin] PATH_INFO: `/v3/users/b66dd2c6e7694191810a310a769dc893` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:02.740082 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f869745-4b6a-47ef-9a7b-8ebb67cf8138 None admin] RBAC: Authorizing `identity:get_user(user_id=b66dd2c6e7694191810a310a769dc893)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:02.741534 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f869745-4b6a-47ef-9a7b-8ebb67cf8138 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:02.742454 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee0666-f1ed-4373-8e64-eed9c86fbab5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:02.743892 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 87/170] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:17:02 2022] GET /identity/v3/users/b66dd2c6e7694191810a310a769dc893 => generated 275 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:02.744905 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee0666-f1ed-4373-8e64-eed9c86fbab5 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:02.745165 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f869745-4b6a-47ef-9a7b-8ebb67cf8138 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:02.864490 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:02.870385 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f869745-4b6a-47ef-9a7b-8ebb67cf8138 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:02.873507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 84/171] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:17:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:17:02.875077 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2fa1e-0047-4cde-97d6-1fc5437bdf3e None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:02.875503 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2fa1e-0047-4cde-97d6-1fc5437bdf3e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:02.875937 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2fa1e-0047-4cde-97d6-1fc5437bdf3e None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:02.877946 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 85/172] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:02.878922 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d2fa1e-0047-4cde-97d6-1fc5437bdf3e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:02.881940 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:02.915337 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b33422-b996-4507-9858-626d8c66f181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:02.916120 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b33422-b996-4507-9858-626d8c66f181 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:02.916460 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b33422-b996-4507-9858-626d8c66f181 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:02.916775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b33422-b996-4507-9858-626d8c66f181 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:02.917598 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b33422-b996-4507-9858-626d8c66f181 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:02.917911 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b33422-b996-4507-9858-626d8c66f181 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:02.919349 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b33422-b996-4507-9858-626d8c66f181 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:02.929320 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 88/173] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 21:17:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:02.930163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b33422-b996-4507-9858-626d8c66f181 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:02.992359 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d2fa1e-0047-4cde-97d6-1fc5437bdf3e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:03.030832 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd77835d-7004-4afa-b399-73924ca1897b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:03.031570 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd77835d-7004-4afa-b399-73924ca1897b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:03.031771 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd77835d-7004-4afa-b399-73924ca1897b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:03.032042 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd77835d-7004-4afa-b399-73924ca1897b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:03.037533 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b33422-b996-4507-9858-626d8c66f181 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:03.071688 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c49aa-57be-4df7-8de2-e497a6b1307a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:03.072662 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c49aa-57be-4df7-8de2-e497a6b1307a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:03.072947 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c49aa-57be-4df7-8de2-e497a6b1307a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:03.073223 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c49aa-57be-4df7-8de2-e497a6b1307a None admin] PATH_INFO: `/v3/projects/c12a755e369e4c55a7d4a855fc16d986` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:03.074469 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90c49aa-57be-4df7-8de2-e497a6b1307a None admin] RBAC: Authorizing `identity:get_project(project_id=c12a755e369e4c55a7d4a855fc16d986)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:03.075930 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90c49aa-57be-4df7-8de2-e497a6b1307a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:03.077669 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 89/174] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:17:02 2022] GET /identity/v3/projects/c12a755e369e4c55a7d4a855fc16d986 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:03.078898 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0c150-78ee-4164-97cd-f8e177a66f71 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:03.079185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0c150-78ee-4164-97cd-f8e177a66f71 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:03.079465 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0c150-78ee-4164-97cd-f8e177a66f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:03.096626 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9cd0c150-78ee-4164-97cd-f8e177a66f71 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:03.097889 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9cd0c150-78ee-4164-97cd-f8e177a66f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:03.106549 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd77835d-7004-4afa-b399-73924ca1897b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:03.108446 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd77835d-7004-4afa-b399-73924ca1897b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:03.110686 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:03.122519 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:03.165965 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 90/175] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:17:03.166856 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0c150-78ee-4164-97cd-f8e177a66f71 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:03.218845 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 86/176] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:17:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:17:03.218845 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547074f5-e5fc-48d6-a0a4-48642096b8c0 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:03.218845 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547074f5-e5fc-48d6-a0a4-48642096b8c0 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:03.218845 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547074f5-e5fc-48d6-a0a4-48642096b8c0 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:03.218845 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 87/177] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:03.218845 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f42717-0fae-4d1e-ab4a-49c286f59a89 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:03.218845 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f42717-0fae-4d1e-ab4a-49c286f59a89 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:03.219487 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f42717-0fae-4d1e-ab4a-49c286f59a89 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:03.220611 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 88/178] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:03.221367 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f42717-0fae-4d1e-ab4a-49c286f59a89 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:03.270452 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0c150-78ee-4164-97cd-f8e177a66f71 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:03.306276 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a8558b-ba4b-4a1c-9639-91b188f63544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:03.307179 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8558b-ba4b-4a1c-9639-91b188f63544 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:03.307462 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8558b-ba4b-4a1c-9639-91b188f63544 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:03.307740 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8558b-ba4b-4a1c-9639-91b188f63544 None admin] PATH_INFO: `/v3/users/1014aa77618f4aaf82cd5198bc2ff5e0` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:03.308957 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a8558b-ba4b-4a1c-9639-91b188f63544 None admin] RBAC: Authorizing `identity:get_user(user_id=1014aa77618f4aaf82cd5198bc2ff5e0)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:03.310470 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a8558b-ba4b-4a1c-9639-91b188f63544 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:03.312139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 91/179] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:17:03 2022] GET /identity/v3/users/1014aa77618f4aaf82cd5198bc2ff5e0 => generated 289 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:03.312923 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a8558b-ba4b-4a1c-9639-91b188f63544 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:03.330718 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f42717-0fae-4d1e-ab4a-49c286f59a89 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:03.332472 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:03.344613 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:03.371929 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc22c52-1ddd-4468-ab02-d869df2f7f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:03.372523 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc22c52-1ddd-4468-ab02-d869df2f7f18 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:03.372798 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc22c52-1ddd-4468-ab02-d869df2f7f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:03.373070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc22c52-1ddd-4468-ab02-d869df2f7f18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:03.373896 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc22c52-1ddd-4468-ab02-d869df2f7f18 None admin] RBAC: Adding query filter params (role.id=566c421659204df185f8b77bf3bb3afc, scope.project.id=c12a755e369e4c55a7d4a855fc16d986, user.id=b66dd2c6e7694191810a310a769dc893) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:03.374209 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc22c52-1ddd-4468-ab02-d869df2f7f18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:03.375681 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc22c52-1ddd-4468-ab02-d869df2f7f18 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:03.384755 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 89/180] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:17:03 2022] GET /identity/v3/role_assignments?user.id=b66dd2c6e7694191810a310a769dc893&scope.project.id=c12a755e369e4c55a7d4a855fc16d986&role.id=566c421659204df185f8b77bf3bb3afc => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:03.384755 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc22c52-1ddd-4468-ab02-d869df2f7f18 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:03.423604 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a8558b-ba4b-4a1c-9639-91b188f63544 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:03.463536 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3a1963-92bb-4f49-9376-f74be756df60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:03.464501 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a1963-92bb-4f49-9376-f74be756df60 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:03.464849 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a1963-92bb-4f49-9376-f74be756df60 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:03.465090 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a1963-92bb-4f49-9376-f74be756df60 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:03.477115 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3a1963-92bb-4f49-9376-f74be756df60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:03.485314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-4d3a1963-92bb-4f49-9376-f74be756df60 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:17:03.485314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:03.485314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:03.485314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:03.485314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:03.485314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:03.485314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:03.485314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:03.485314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:03.485314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:03.485314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:03.485314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:03.485314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:03.485314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:03.485314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:03.485314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:03.486469 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:03.486469 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:03.486469 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:03.486469 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:03.486469 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:03.486469 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:03.486469 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:03.486469 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:03.486469 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:03.486469 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:03.486469 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:03.486469 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:17:03.486469 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:03.486469 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:03.486469 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:03.486469 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:03.487257 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:03.487257 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:03.487257 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:03.487257 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:03.487257 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:03.487257 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:03.487257 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:03.487257 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:03.487257 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:03.487257 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:03.487257 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:17:03.487257 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:17:03.487257 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 92/181] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 21:17:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:03.487918 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbacc527-245f-45b3-8c4b-f56c29bc1981 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:03.488582 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbacc527-245f-45b3-8c4b-f56c29bc1981 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:03.488582 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbacc527-245f-45b3-8c4b-f56c29bc1981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:03.504416 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc22c52-1ddd-4468-ab02-d869df2f7f18 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:03.511421 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-cbacc527-245f-45b3-8c4b-f56c29bc1981 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:03.512821 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cbacc527-245f-45b3-8c4b-f56c29bc1981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:03.544589 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a13e0c-8e5b-43db-add3-45419e0a442a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:03.545474 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a13e0c-8e5b-43db-add3-45419e0a442a None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:03.545751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a13e0c-8e5b-43db-add3-45419e0a442a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:03.546026 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a13e0c-8e5b-43db-add3-45419e0a442a None admin] PATH_INFO: `/v3/projects/da0a63de1c7f4c898f8c9db9d8992d51` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:03.547533 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a13e0c-8e5b-43db-add3-45419e0a442a None admin] RBAC: Authorizing `identity:get_project(project_id=da0a63de1c7f4c898f8c9db9d8992d51)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:03.548986 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a13e0c-8e5b-43db-add3-45419e0a442a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:03.551247 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 90/182] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:17:03 2022] GET /identity/v3/projects/da0a63de1c7f4c898f8c9db9d8992d51 => generated 312 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:03.552633 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b547723-1e27-4046-ba37-aef1615e228c None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:03.552914 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b547723-1e27-4046-ba37-aef1615e228c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:03.553185 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b547723-1e27-4046-ba37-aef1615e228c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:03.574779 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-6b547723-1e27-4046-ba37-aef1615e228c None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:03.575965 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6b547723-1e27-4046-ba37-aef1615e228c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:03.600033 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 93/183] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:17:03.600806 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbacc527-245f-45b3-8c4b-f56c29bc1981 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:03.615868 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:03.625390 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:03.664076 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 91/184] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:17:03.664945 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b547723-1e27-4046-ba37-aef1615e228c None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:03.725445 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbacc527-245f-45b3-8c4b-f56c29bc1981 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:03.765270 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa923728-1923-4d38-8747-9cad77b010dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:03.766732 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa923728-1923-4d38-8747-9cad77b010dd None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:03.767187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa923728-1923-4d38-8747-9cad77b010dd None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:03.767586 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa923728-1923-4d38-8747-9cad77b010dd None admin] PATH_INFO: `/v3/projects/c12a755e369e4c55a7d4a855fc16d986` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:03.769564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa923728-1923-4d38-8747-9cad77b010dd None admin] RBAC: Authorizing `identity:get_project(project_id=c12a755e369e4c55a7d4a855fc16d986)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:03.771653 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b547723-1e27-4046-ba37-aef1615e228c None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:03.772094 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa923728-1923-4d38-8747-9cad77b010dd None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:03.775247 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 94/185] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:17:03 2022] GET /identity/v3/projects/c12a755e369e4c55a7d4a855fc16d986 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:03.775247 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa923728-1923-4d38-8747-9cad77b010dd None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:03.816001 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c87baf-e46a-4c5c-a65e-b36fca50729a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:03.816689 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c87baf-e46a-4c5c-a65e-b36fca50729a None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:03.816967 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c87baf-e46a-4c5c-a65e-b36fca50729a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:03.817985 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c87baf-e46a-4c5c-a65e-b36fca50729a None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:03.817985 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c87baf-e46a-4c5c-a65e-b36fca50729a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:03.818308 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c87baf-e46a-4c5c-a65e-b36fca50729a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:03.819703 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c87baf-e46a-4c5c-a65e-b36fca50729a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:03.828692 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 92/186] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:17:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:03.829544 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c87baf-e46a-4c5c-a65e-b36fca50729a None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:03.865727 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:03.879104 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:03.894090 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa923728-1923-4d38-8747-9cad77b010dd None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:03.928851 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a48d7-19bf-4877-94b0-d038fb3ba68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:03.929521 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a48d7-19bf-4877-94b0-d038fb3ba68c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:03.929822 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a48d7-19bf-4877-94b0-d038fb3ba68c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:03.930096 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a48d7-19bf-4877-94b0-d038fb3ba68c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:03.931096 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3a48d7-19bf-4877-94b0-d038fb3ba68c None admin] RBAC: Adding query filter params (role.id=70699fb0eae7441999835fa3b50350b3, scope.project.id=da0a63de1c7f4c898f8c9db9d8992d51, user.id=b66dd2c6e7694191810a310a769dc893) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:03.931958 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3a48d7-19bf-4877-94b0-d038fb3ba68c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:03.933429 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3a48d7-19bf-4877-94b0-d038fb3ba68c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:03.939484 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 95/187] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:17:03 2022] GET /identity/v3/role_assignments?user.id=b66dd2c6e7694191810a310a769dc893&scope.project.id=da0a63de1c7f4c898f8c9db9d8992d51&role.id=70699fb0eae7441999835fa3b50350b3 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:03.940273 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a48d7-19bf-4877-94b0-d038fb3ba68c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:03.942703 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c87baf-e46a-4c5c-a65e-b36fca50729a None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:03.982015 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1349710c-b7d0-4919-a092-a15206675427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:03.982902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1349710c-b7d0-4919-a092-a15206675427 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:03.983192 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1349710c-b7d0-4919-a092-a15206675427 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:03.983466 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1349710c-b7d0-4919-a092-a15206675427 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:03.984801 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1349710c-b7d0-4919-a092-a15206675427 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:03.986391 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1349710c-b7d0-4919-a092-a15206675427 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:03.988279 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 93/188] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:17:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:03.988936 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1349710c-b7d0-4919-a092-a15206675427 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:04.071007 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a48d7-19bf-4877-94b0-d038fb3ba68c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:04.127014 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:04.130971 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f702358-56c5-4088-a95f-d6558ee53486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:04.132222 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f702358-56c5-4088-a95f-d6558ee53486 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:04.132513 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f702358-56c5-4088-a95f-d6558ee53486 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:04.132933 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f702358-56c5-4088-a95f-d6558ee53486 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:04.135047 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1349710c-b7d0-4919-a092-a15206675427 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:04.149945 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f702358-56c5-4088-a95f-d6558ee53486 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:04.159647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-5f702358-56c5-4088-a95f-d6558ee53486 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:17:04.159647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:04.159647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:04.159647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:04.159647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:04.159647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:04.159647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:04.159647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:04.159647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:04.159647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:04.159647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:04.159647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:04.159647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:04.159647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:04.159647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:04.159647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:04.160872 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:04.160872 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:04.160872 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:04.160872 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:04.160872 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:04.160872 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:04.160872 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:04.160872 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:04.160872 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:04.160872 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:04.160872 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:04.160872 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:17:04.160872 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:04.160872 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:04.160872 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:04.160872 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:04.161841 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:04.161841 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:04.161841 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:04.161841 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:04.161841 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:04.161841 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:04.161841 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:04.161841 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:04.161841 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:04.161841 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:04.161841 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:17:04.161841 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:17:04.163495 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 96/189] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 21:17:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:04.164522 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f702358-56c5-4088-a95f-d6558ee53486 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:04.185175 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:04.197152 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b8615-4b8c-4ac0-aed2-aff1c8ae665e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:04.198688 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b8615-4b8c-4ac0-aed2-aff1c8ae665e None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:04.198981 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b8615-4b8c-4ac0-aed2-aff1c8ae665e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:04.199258 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b8615-4b8c-4ac0-aed2-aff1c8ae665e None admin] PATH_INFO: `/v3/projects/c12a755e369e4c55a7d4a855fc16d986/users/1014aa77618f4aaf82cd5198bc2ff5e0/roles/34a7a66da3654831a115d77e40a989da` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:04.201311 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b8615-4b8c-4ac0-aed2-aff1c8ae665e None admin] RBAC: Authorizing `identity:create_grant(project_id=c12a755e369e4c55a7d4a855fc16d986, user_id=1014aa77618f4aaf82cd5198bc2ff5e0, role_id=34a7a66da3654831a115d77e40a989da)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:04.202887 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b8615-4b8c-4ac0-aed2-aff1c8ae665e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:04.220007 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 94/190] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 21:17:03 2022] PUT /identity/v3/projects/c12a755e369e4c55a7d4a855fc16d986/users/1014aa77618f4aaf82cd5198bc2ff5e0/roles/34a7a66da3654831a115d77e40a989da => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:17:04.220851 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b8615-4b8c-4ac0-aed2-aff1c8ae665e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:04.331542 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f702358-56c5-4088-a95f-d6558ee53486 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:04.350841 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b8615-4b8c-4ac0-aed2-aff1c8ae665e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:04.375934 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a14ec-c783-467b-aaa3-25ae003d2b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:04.376944 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a14ec-c783-467b-aaa3-25ae003d2b6a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:04.377287 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a14ec-c783-467b-aaa3-25ae003d2b6a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:04.377691 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a14ec-c783-467b-aaa3-25ae003d2b6a None admin] PATH_INFO: `/v3/users/a618269d48c34c1da396412788af1a3d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:04.398024 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a14ec-c783-467b-aaa3-25ae003d2b6a None admin] RBAC: Authorizing `identity:get_user(user_id=a618269d48c34c1da396412788af1a3d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:04.398532 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b043365-c6c0-426b-af6a-54717e7cf9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:04.399312 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b043365-c6c0-426b-af6a-54717e7cf9af None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:04.399688 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b043365-c6c0-426b-af6a-54717e7cf9af None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:04.399991 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a14ec-c783-467b-aaa3-25ae003d2b6a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:04.400086 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b043365-c6c0-426b-af6a-54717e7cf9af None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:04.401440 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b043365-c6c0-426b-af6a-54717e7cf9af None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:04.403026 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b043365-c6c0-426b-af6a-54717e7cf9af None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:04.403938 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 97/191] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:17:04 2022] GET /identity/v3/users/a618269d48c34c1da396412788af1a3d => generated 283 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:04.404589 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a14ec-c783-467b-aaa3-25ae003d2b6a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:04.435017 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 95/192] 10.222.0.65 () {60 vars in 1132 bytes} [Thu May 19 21:17:04 2022] POST /identity/v3/users => generated 297 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:17:04.438289 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b043365-c6c0-426b-af6a-54717e7cf9af None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:04.462593 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:04.548245 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:04.630282 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a14ec-c783-467b-aaa3-25ae003d2b6a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:04.656823 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b043365-c6c0-426b-af6a-54717e7cf9af None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:04.686127 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f00a0-c7e1-469b-bc16-d291e8ebd4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:04.686886 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f00a0-c7e1-469b-bc16-d291e8ebd4f5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:04.687171 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f00a0-c7e1-469b-bc16-d291e8ebd4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:04.687451 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f00a0-c7e1-469b-bc16-d291e8ebd4f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:04.688799 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f00a0-c7e1-469b-bc16-d291e8ebd4f5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:04.689128 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f00a0-c7e1-469b-bc16-d291e8ebd4f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:04.690796 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f00a0-c7e1-469b-bc16-d291e8ebd4f5 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:04.702916 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 98/193] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:17:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:04.709019 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f00a0-c7e1-469b-bc16-d291e8ebd4f5 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:04.710799 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff36b85-8888-4d92-bb7e-74c3813d917f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:04.711509 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff36b85-8888-4d92-bb7e-74c3813d917f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:04.711965 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff36b85-8888-4d92-bb7e-74c3813d917f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:04.713039 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff36b85-8888-4d92-bb7e-74c3813d917f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:04.824223 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff36b85-8888-4d92-bb7e-74c3813d917f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:04.826128 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff36b85-8888-4d92-bb7e-74c3813d917f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:04.847882 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:04.854425 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f00a0-c7e1-469b-bc16-d291e8ebd4f5 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:04.909491 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:04.921575 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e688e1f-47e4-4771-8b34-c2fa204c5016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:04.922517 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e688e1f-47e4-4771-8b34-c2fa204c5016 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:04.922800 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e688e1f-47e4-4771-8b34-c2fa204c5016 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:04.923073 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e688e1f-47e4-4771-8b34-c2fa204c5016 None admin] PATH_INFO: `/v3/users/a618269d48c34c1da396412788af1a3d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:04.924485 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e688e1f-47e4-4771-8b34-c2fa204c5016 None admin] RBAC: Authorizing `identity:get_user(user_id=a618269d48c34c1da396412788af1a3d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:04.926801 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e688e1f-47e4-4771-8b34-c2fa204c5016 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:04.929446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 99/194] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:17:04 2022] GET /identity/v3/users/a618269d48c34c1da396412788af1a3d => generated 283 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:04.935083 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e688e1f-47e4-4771-8b34-c2fa204c5016 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:04.986597 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 96/195] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:17:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:17:04.992574 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff36b85-8888-4d92-bb7e-74c3813d917f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:05.096191 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e688e1f-47e4-4771-8b34-c2fa204c5016 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:05.145248 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e99a96-317c-41d3-94eb-45cd22dfd3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:05.146135 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e99a96-317c-41d3-94eb-45cd22dfd3bf None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:05.146538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e99a96-317c-41d3-94eb-45cd22dfd3bf None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:05.146909 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e99a96-317c-41d3-94eb-45cd22dfd3bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:05.195695 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff36b85-8888-4d92-bb7e-74c3813d917f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:05.209521 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:05.239949 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd07e6-e726-419c-b650-4d0e97bc4142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:05.241267 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd07e6-e726-419c-b650-4d0e97bc4142 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:05.241767 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd07e6-e726-419c-b650-4d0e97bc4142 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:05.242362 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd07e6-e726-419c-b650-4d0e97bc4142 None admin] PATH_INFO: `/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:05.244422 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cd07e6-e726-419c-b650-4d0e97bc4142 None admin] RBAC: Authorizing `identity:get_project(project_id=1264ceb6aa114fbf81b3d48cd8874c8f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:05.246168 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cd07e6-e726-419c-b650-4d0e97bc4142 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:05.248298 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 97/196] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:17:04 2022] GET /identity/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:17:05.260204 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd07e6-e726-419c-b650-4d0e97bc4142 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:05.276782 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:05.284906 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e99a96-317c-41d3-94eb-45cd22dfd3bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:05.286567 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e99a96-317c-41d3-94eb-45cd22dfd3bf None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:05.388984 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd07e6-e726-419c-b650-4d0e97bc4142 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:05.426528 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d0216-c1d1-42f4-893f-499679ee0687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:05.427583 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d0216-c1d1-42f4-893f-499679ee0687 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:05.428057 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d0216-c1d1-42f4-893f-499679ee0687 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:05.428518 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d0216-c1d1-42f4-893f-499679ee0687 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:05.429546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d0216-c1d1-42f4-893f-499679ee0687 None admin] RBAC: Adding query filter params (role.id=2b714f2e5c6c469f8fa037bda1ed16f6, scope.project.id=1264ceb6aa114fbf81b3d48cd8874c8f, user.id=a618269d48c34c1da396412788af1a3d) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:05.430046 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d0216-c1d1-42f4-893f-499679ee0687 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:05.431804 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d0216-c1d1-42f4-893f-499679ee0687 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:05.438617 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 98/197] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:17:05 2022] GET /identity/v3/role_assignments?user.id=a618269d48c34c1da396412788af1a3d&scope.project.id=1264ceb6aa114fbf81b3d48cd8874c8f&role.id=2b714f2e5c6c469f8fa037bda1ed16f6 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:17:05.503848 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 100/198] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:17:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:17:05.510670 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d0216-c1d1-42f4-893f-499679ee0687 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:05.598736 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:05.673155 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d0216-c1d1-42f4-893f-499679ee0687 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:05.710078 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3752b8c6-a036-4fb0-9cdf-a1b5a7155fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:05.710992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752b8c6-a036-4fb0-9cdf-a1b5a7155fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:05.711513 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752b8c6-a036-4fb0-9cdf-a1b5a7155fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:05.711513 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752b8c6-a036-4fb0-9cdf-a1b5a7155fc0 None admin] PATH_INFO: `/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:05.712986 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3752b8c6-a036-4fb0-9cdf-a1b5a7155fc0 None admin] RBAC: Authorizing `identity:get_project(project_id=1264ceb6aa114fbf81b3d48cd8874c8f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:05.714291 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3752b8c6-a036-4fb0-9cdf-a1b5a7155fc0 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:05.716419 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 99/199] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:17:05 2022] GET /identity/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:17:05.722136 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e99a96-317c-41d3-94eb-45cd22dfd3bf None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:05.845094 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e99a96-317c-41d3-94eb-45cd22dfd3bf None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:05.858629 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:05.902738 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed628fa-43e6-4295-bcc6-1a38a44973e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:05.903409 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed628fa-43e6-4295-bcc6-1a38a44973e1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:05.903701 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed628fa-43e6-4295-bcc6-1a38a44973e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:05.903980 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed628fa-43e6-4295-bcc6-1a38a44973e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:05.904871 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed628fa-43e6-4295-bcc6-1a38a44973e1 None admin] RBAC: Adding query filter params (role.id=566c421659204df185f8b77bf3bb3afc, scope.project.id=1264ceb6aa114fbf81b3d48cd8874c8f, user.id=a618269d48c34c1da396412788af1a3d) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:05.905187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed628fa-43e6-4295-bcc6-1a38a44973e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:05.906730 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed628fa-43e6-4295-bcc6-1a38a44973e1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:05.912767 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 101/200] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:17:05 2022] GET /identity/v3/role_assignments?user.id=a618269d48c34c1da396412788af1a3d&scope.project.id=1264ceb6aa114fbf81b3d48cd8874c8f&role.id=566c421659204df185f8b77bf3bb3afc => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:06.889189 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff06ef-6235-48ba-a52f-b6e590761134 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:06.890076 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff06ef-6235-48ba-a52f-b6e590761134 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:06.890538 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff06ef-6235-48ba-a52f-b6e590761134 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:06.892568 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 100/201] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:06.898081 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d945f3-da61-4f48-820c-7bd3a08b63ba None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:06.898951 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d945f3-da61-4f48-820c-7bd3a08b63ba None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:06.899408 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d945f3-da61-4f48-820c-7bd3a08b63ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:06.928345 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-52d945f3-da61-4f48-820c-7bd3a08b63ba None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:06.929785 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-52d945f3-da61-4f48-820c-7bd3a08b63ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:07.017076 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 102/202] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:17:07.050395 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff06ef-6235-48ba-a52f-b6e590761134 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:07.187355 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff06ef-6235-48ba-a52f-b6e590761134 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:07.214745 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:07.245509 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5744136-d476-4208-be48-7455c697595b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:07.246396 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5744136-d476-4208-be48-7455c697595b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:07.246684 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5744136-d476-4208-be48-7455c697595b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:07.246964 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5744136-d476-4208-be48-7455c697595b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:07.268960 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5744136-d476-4208-be48-7455c697595b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:07.277177 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-b5744136-d476-4208-be48-7455c697595b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 21:17:07.277177 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:07.277177 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:07.277177 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:07.277177 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:07.277177 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:07.277177 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:07.277177 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:07.277177 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:07.277177 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:07.277177 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:07.277177 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:07.277177 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:07.277177 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:07.277177 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:07.277177 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:07.278347 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:07.278347 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:07.278347 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:07.278347 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:07.278347 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:07.278347 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:07.278347 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:07.278347 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:07.278347 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:07.278347 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:07.278347 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:07.278347 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:07.278347 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:07.278347 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:07.278347 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:07.278347 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:07.279284 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:07.279284 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:07.279284 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:07.279284 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:07.279284 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:07.279284 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:07.279284 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:07.279284 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:07.279284 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:07.279284 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:07.279284 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 21:17:07.279284 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:07.279284 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 101/203] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:17:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:07.286588 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d945f3-da61-4f48-820c-7bd3a08b63ba None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:07.414971 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:07.425320 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d945f3-da61-4f48-820c-7bd3a08b63ba None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:07.493222 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef97c20-2a26-4c50-a6c6-fe4ba805b081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:07.494125 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704f09e-d0e1-4f0c-a281-7120408ef484 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:07.494125 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704f09e-d0e1-4f0c-a281-7120408ef484 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:07.494125 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704f09e-d0e1-4f0c-a281-7120408ef484 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:07.495019 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef97c20-2a26-4c50-a6c6-fe4ba805b081 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:07.495345 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef97c20-2a26-4c50-a6c6-fe4ba805b081 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:07.495615 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 102/204] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:17:07.495712 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef97c20-2a26-4c50-a6c6-fe4ba805b081 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:07.496465 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef97c20-2a26-4c50-a6c6-fe4ba805b081 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:07.496775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef97c20-2a26-4c50-a6c6-fe4ba805b081 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:07.498303 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef97c20-2a26-4c50-a6c6-fe4ba805b081 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:07.506568 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f226125-f5b9-4510-b4ad-c99c507779b7 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:07.506885 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f226125-f5b9-4510-b4ad-c99c507779b7 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:07.507269 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f226125-f5b9-4510-b4ad-c99c507779b7 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:07.511375 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 103/205] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:07.512739 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736366b9-f841-40c7-b269-2f4d2903f2bb None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:07.513029 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736366b9-f841-40c7-b269-2f4d2903f2bb None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:07.513308 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736366b9-f841-40c7-b269-2f4d2903f2bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:07.517042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 103/206] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 21:17:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:07.518436 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6d56b-7856-4d23-b1a6-ef7fbb991acb None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:07.518732 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6d56b-7856-4d23-b1a6-ef7fbb991acb None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:07.519014 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6d56b-7856-4d23-b1a6-ef7fbb991acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:07.535429 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-736366b9-f841-40c7-b269-2f4d2903f2bb None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:07.537720 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-736366b9-f841-40c7-b269-2f4d2903f2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:07.543585 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d4f6d56b-7856-4d23-b1a6-ef7fbb991acb None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:07.545134 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d4f6d56b-7856-4d23-b1a6-ef7fbb991acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:07.630836 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 104/207] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:17:07.631885 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736366b9-f841-40c7-b269-2f4d2903f2bb None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:07.671646 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 104/208] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:17:07.672969 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f6d56b-7856-4d23-b1a6-ef7fbb991acb None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:07.790353 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:07.791000 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:07.796008 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736366b9-f841-40c7-b269-2f4d2903f2bb None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:07.814382 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f6d56b-7856-4d23-b1a6-ef7fbb991acb None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:07.847128 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad9e17d-ea58-4cb3-8f91-051e9b7e3bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:07.848037 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9e17d-ea58-4cb3-8f91-051e9b7e3bde None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:07.848314 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9e17d-ea58-4cb3-8f91-051e9b7e3bde None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:07.849149 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9e17d-ea58-4cb3-8f91-051e9b7e3bde None admin] PATH_INFO: `/v3/users/1014aa77618f4aaf82cd5198bc2ff5e0` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:07.850416 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad9e17d-ea58-4cb3-8f91-051e9b7e3bde None admin] RBAC: Authorizing `identity:get_user(user_id=1014aa77618f4aaf82cd5198bc2ff5e0)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:07.851998 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad9e17d-ea58-4cb3-8f91-051e9b7e3bde None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:07.860661 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 105/209] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:17:07 2022] GET /identity/v3/users/1014aa77618f4aaf82cd5198bc2ff5e0 => generated 289 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:07.860661 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad9e17d-ea58-4cb3-8f91-051e9b7e3bde None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:07.874796 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4130cc-a45f-4c6a-8bb8-43dd0e19818b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:07.876067 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4130cc-a45f-4c6a-8bb8-43dd0e19818b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:07.876467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4130cc-a45f-4c6a-8bb8-43dd0e19818b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:07.876774 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4130cc-a45f-4c6a-8bb8-43dd0e19818b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:07.878516 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4130cc-a45f-4c6a-8bb8-43dd0e19818b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:07.880605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4130cc-a45f-4c6a-8bb8-43dd0e19818b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:07.887502 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 105/210] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:17:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:07.887502 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4130cc-a45f-4c6a-8bb8-43dd0e19818b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:07.999963 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4130cc-a45f-4c6a-8bb8-43dd0e19818b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:08.030021 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad9e17d-ea58-4cb3-8f91-051e9b7e3bde None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:08.046402 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627c76b-a5d2-4c50-b2ce-4130bf37c446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:08.047689 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627c76b-a5d2-4c50-b2ce-4130bf37c446 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:08.048675 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627c76b-a5d2-4c50-b2ce-4130bf37c446 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:08.049158 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627c76b-a5d2-4c50-b2ce-4130bf37c446 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:08.078659 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:08.083989 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48908853-a2a2-41f8-b367-05ed1ab9b288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:08.085067 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48908853-a2a2-41f8-b367-05ed1ab9b288 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:08.085421 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48908853-a2a2-41f8-b367-05ed1ab9b288 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:08.085786 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48908853-a2a2-41f8-b367-05ed1ab9b288 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:08.104152 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48908853-a2a2-41f8-b367-05ed1ab9b288 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:08.114864 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-48908853-a2a2-41f8-b367-05ed1ab9b288 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 21:17:08.114864 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:08.114864 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:08.114864 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:08.114864 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:08.114864 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:08.114864 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:08.114864 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:08.114864 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:08.114864 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:08.114864 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:08.114864 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:08.114864 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:08.114864 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:08.114864 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:08.114864 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:08.115827 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:08.115827 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:08.115827 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:08.115827 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:08.115827 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:08.115827 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:08.115827 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:08.115827 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:08.115827 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:08.115827 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:08.115827 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:08.115827 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:08.115827 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:08.115827 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:08.115827 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:08.115827 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:08.116618 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:08.116618 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:08.116618 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:08.116618 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:08.116618 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:08.116618 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:08.116618 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:08.116618 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:08.116618 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:08.116618 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:08.116618 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 21:17:08.116618 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:08.117440 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 106/211] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:17:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:08.122648 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48908853-a2a2-41f8-b367-05ed1ab9b288 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:08.151630 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4627c76b-a5d2-4c50-b2ce-4130bf37c446 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:08.153140 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4627c76b-a5d2-4c50-b2ce-4130bf37c446 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:08.168915 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:08.272331 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48908853-a2a2-41f8-b367-05ed1ab9b288 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:08.277605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 106/212] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:17:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:17:08.278518 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627c76b-a5d2-4c50-b2ce-4130bf37c446 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:08.317203 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052f889b-8353-4b35-8127-d9370d6c9e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:08.318164 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f889b-8353-4b35-8127-d9370d6c9e8a None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:08.318640 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f889b-8353-4b35-8127-d9370d6c9e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:08.319052 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f889b-8353-4b35-8127-d9370d6c9e8a None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:08.319903 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052f889b-8353-4b35-8127-d9370d6c9e8a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:08.321762 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052f889b-8353-4b35-8127-d9370d6c9e8a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:08.344752 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:08.350220 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 107/213] 10.222.0.65 () {60 vars in 1132 bytes} [Thu May 19 21:17:08 2022] POST /identity/v3/users => generated 297 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:17:08.352056 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052f889b-8353-4b35-8127-d9370d6c9e8a None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:08.433409 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627c76b-a5d2-4c50-b2ce-4130bf37c446 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:08.483224 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:08.500015 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e69a0-b635-493e-82ea-26c338e9d9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:08.501878 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e69a0-b635-493e-82ea-26c338e9d9cd None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:08.503633 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e69a0-b635-493e-82ea-26c338e9d9cd None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:08.504328 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e69a0-b635-493e-82ea-26c338e9d9cd None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:08.506221 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2e69a0-b635-493e-82ea-26c338e9d9cd None admin] RBAC: Adding query filter params (name=member) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:08.506928 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2e69a0-b635-493e-82ea-26c338e9d9cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:08.509739 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2e69a0-b635-493e-82ea-26c338e9d9cd None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:08.523984 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 107/214] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 21:17:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:08.531714 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e69a0-b635-493e-82ea-26c338e9d9cd None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:08.546756 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052f889b-8353-4b35-8127-d9370d6c9e8a None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:08.664532 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ab5c75-3386-4f6a-a8c4-9c2ce484762f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:08.665611 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab5c75-3386-4f6a-a8c4-9c2ce484762f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:08.665987 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab5c75-3386-4f6a-a8c4-9c2ce484762f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:08.666390 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab5c75-3386-4f6a-a8c4-9c2ce484762f None admin] PATH_INFO: `/v3/projects/c12a755e369e4c55a7d4a855fc16d986` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:08.667904 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ab5c75-3386-4f6a-a8c4-9c2ce484762f None admin] RBAC: Authorizing `identity:get_project(project_id=c12a755e369e4c55a7d4a855fc16d986)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:08.669596 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ab5c75-3386-4f6a-a8c4-9c2ce484762f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:08.671616 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 108/215] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:17:08 2022] GET /identity/v3/projects/c12a755e369e4c55a7d4a855fc16d986 => generated 298 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:08.675462 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e69a0-b635-493e-82ea-26c338e9d9cd None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:08.678221 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e9b97-b379-4507-a9ae-d4a980eb4080 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:08.678727 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e9b97-b379-4507-a9ae-d4a980eb4080 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:08.679049 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e9b97-b379-4507-a9ae-d4a980eb4080 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:08.686291 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 109/216] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:08 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:08.686291 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e9b97-b379-4507-a9ae-d4a980eb4080 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:08.701091 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:08.726961 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97722799-159e-47dd-afa0-3174db61afc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:08.728474 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97722799-159e-47dd-afa0-3174db61afc7 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:08.729073 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97722799-159e-47dd-afa0-3174db61afc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:08.729510 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97722799-159e-47dd-afa0-3174db61afc7 None admin] PATH_INFO: `/v3/users/ddf67c24de3243b2b6cdb4db14ed1ad5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:08.746714 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97722799-159e-47dd-afa0-3174db61afc7 None admin] RBAC: Authorizing `identity:get_user(user_id=ddf67c24de3243b2b6cdb4db14ed1ad5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:08.748486 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97722799-159e-47dd-afa0-3174db61afc7 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:08.750663 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 108/217] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:17:08 2022] GET /identity/v3/users/ddf67c24de3243b2b6cdb4db14ed1ad5 => generated 297 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:08.752453 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ea812-379d-4cb1-8f1c-570efb26676f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:08.757055 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ea812-379d-4cb1-8f1c-570efb26676f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:08.757333 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ea812-379d-4cb1-8f1c-570efb26676f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:08.775169 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f16ea812-379d-4cb1-8f1c-570efb26676f None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:08.776462 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f16ea812-379d-4cb1-8f1c-570efb26676f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:08.838389 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e9b97-b379-4507-a9ae-d4a980eb4080 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:08.877652 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 109/218] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:17:08.880898 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ea812-379d-4cb1-8f1c-570efb26676f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:08.898745 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422acda-196f-4f1a-a62b-39940c2ed708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:08.899262 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422acda-196f-4f1a-a62b-39940c2ed708 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:08.899838 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422acda-196f-4f1a-a62b-39940c2ed708 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:08.900447 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422acda-196f-4f1a-a62b-39940c2ed708 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:08.901954 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f422acda-196f-4f1a-a62b-39940c2ed708 None admin] RBAC: Adding query filter params (role.id=34a7a66da3654831a115d77e40a989da, scope.project.id=c12a755e369e4c55a7d4a855fc16d986, user.id=1014aa77618f4aaf82cd5198bc2ff5e0) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:08.902481 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f422acda-196f-4f1a-a62b-39940c2ed708 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:08.906894 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f422acda-196f-4f1a-a62b-39940c2ed708 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:08.924366 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 110/219] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:17:08 2022] GET /identity/v3/role_assignments?user.id=1014aa77618f4aaf82cd5198bc2ff5e0&scope.project.id=c12a755e369e4c55a7d4a855fc16d986&role.id=34a7a66da3654831a115d77e40a989da => generated 608 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:08.925869 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7dc70-6016-4f72-b1e6-c8135631b14c None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:08.926408 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7dc70-6016-4f72-b1e6-c8135631b14c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:08.926688 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7dc70-6016-4f72-b1e6-c8135631b14c None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:08.928912 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 111/220] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:08.934787 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa7dc70-6016-4f72-b1e6-c8135631b14c None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:08.962368 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:09.048500 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ea812-379d-4cb1-8f1c-570efb26676f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:09.051327 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:09.095596 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c02f8-fa82-45cf-9814-a462301ce317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:09.096302 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa7dc70-6016-4f72-b1e6-c8135631b14c None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:09.096782 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c02f8-fa82-45cf-9814-a462301ce317 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:09.097094 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c02f8-fa82-45cf-9814-a462301ce317 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:09.097562 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c02f8-fa82-45cf-9814-a462301ce317 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:09.137291 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a01707-f40c-416c-8169-908e4a756c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:09.138453 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a01707-f40c-416c-8169-908e4a756c5a None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:09.138906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a01707-f40c-416c-8169-908e4a756c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:09.139325 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a01707-f40c-416c-8169-908e4a756c5a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:09.152667 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a01707-f40c-416c-8169-908e4a756c5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:09.161218 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-02a01707-f40c-416c-8169-908e4a756c5a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:17:09.161218 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:09.161218 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:09.161218 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:09.161218 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:09.161218 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:09.161218 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:09.161218 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:09.161218 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:09.161218 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:09.161218 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:09.161218 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:09.161218 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:09.161218 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:09.161218 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:09.161218 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:09.162207 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:09.162207 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:09.162207 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:09.162207 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:09.162207 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:09.162207 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:09.162207 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:09.162207 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:09.162207 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:09.162207 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:09.162207 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:09.162207 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:09.162207 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:09.162207 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:09.162207 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:09.162207 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:09.163432 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:09.163432 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:09.163432 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:09.163432 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:09.163432 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:09.163432 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:09.163432 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:09.163432 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:09.163432 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:09.163432 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:09.163432 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:17:09.163432 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:09.165835 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 112/221] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 21:17:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:09.167875 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb8533-ef85-451e-a697-ae89bdfb62bb None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:09.168316 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb8533-ef85-451e-a697-ae89bdfb62bb None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:09.168733 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb8533-ef85-451e-a697-ae89bdfb62bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:09.188628 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-dceb8533-ef85-451e-a697-ae89bdfb62bb None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:09.190186 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-dceb8533-ef85-451e-a697-ae89bdfb62bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:09.203828 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615c02f8-fa82-45cf-9814-a462301ce317 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:09.206020 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615c02f8-fa82-45cf-9814-a462301ce317 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:09.270211 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 113/222] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:17:09.271953 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb8533-ef85-451e-a697-ae89bdfb62bb None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:09.306716 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:09.350275 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 110/223] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:17:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:17:09.351187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c02f8-fa82-45cf-9814-a462301ce317 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:09.356158 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:09.414058 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb8533-ef85-451e-a697-ae89bdfb62bb None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:09.478672 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c88da-b211-49a2-a713-5f99b1b9f3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:09.479166 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c88da-b211-49a2-a713-5f99b1b9f3f2 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:09.479443 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c88da-b211-49a2-a713-5f99b1b9f3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:09.479714 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c88da-b211-49a2-a713-5f99b1b9f3f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:09.480862 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54c88da-b211-49a2-a713-5f99b1b9f3f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:09.481104 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54c88da-b211-49a2-a713-5f99b1b9f3f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:09.485126 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54c88da-b211-49a2-a713-5f99b1b9f3f2 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:09.501337 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 114/224] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:17:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:09.502289 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c88da-b211-49a2-a713-5f99b1b9f3f2 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:09.523707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c02f8-fa82-45cf-9814-a462301ce317 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:09.579696 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e660a-9cad-4cc8-8fb9-bbb3d4fb88a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:09.581315 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e660a-9cad-4cc8-8fb9-bbb3d4fb88a5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:09.581748 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e660a-9cad-4cc8-8fb9-bbb3d4fb88a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:09.582163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e660a-9cad-4cc8-8fb9-bbb3d4fb88a5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:09.598572 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e660a-9cad-4cc8-8fb9-bbb3d4fb88a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:09.610310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-f45e660a-9cad-4cc8-8fb9-bbb3d4fb88a5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:17:09.610310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:09.610310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:09.610310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:09.610310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:09.610310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:09.610310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:09.610310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:09.610310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:09.610310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:09.610310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:09.610310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:09.610310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:09.610310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:09.610310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:09.610310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:09.611310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:09.611310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:09.611310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:09.611310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:09.611310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:09.611310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:09.611310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:09.611310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:09.611310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:09.611310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:09.611310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:09.611310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:17:09.611310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:09.611310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:09.611310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:09.611310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:09.612885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:09.612885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:09.612885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:09.612885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:09.612885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:09.612885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:09.612885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:09.612885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:09.612885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:09.612885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:09.612885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:17:09.612885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:17:09.616024 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 111/225] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 21:17:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:09.617108 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e660a-9cad-4cc8-8fb9-bbb3d4fb88a5 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:09.625907 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:09.674860 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c88da-b211-49a2-a713-5f99b1b9f3f2 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:09.718139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:09.730845 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755d673-4578-4f5a-8726-2c31c4f8f629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:09.731832 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755d673-4578-4f5a-8726-2c31c4f8f629 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:09.732163 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755d673-4578-4f5a-8726-2c31c4f8f629 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:09.732953 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755d673-4578-4f5a-8726-2c31c4f8f629 None admin] PATH_INFO: `/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:09.734580 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c755d673-4578-4f5a-8726-2c31c4f8f629 None admin] RBAC: Authorizing `identity:get_project(project_id=1264ceb6aa114fbf81b3d48cd8874c8f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:09.739205 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c755d673-4578-4f5a-8726-2c31c4f8f629 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:09.743095 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 115/226] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:17:09 2022] GET /identity/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f => generated 302 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:09.750096 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755d673-4578-4f5a-8726-2c31c4f8f629 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:09.791618 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e660a-9cad-4cc8-8fb9-bbb3d4fb88a5 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:09.847905 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946f6ea-29b1-4f0f-a62a-5e90eaed138f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:09.848968 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946f6ea-29b1-4f0f-a62a-5e90eaed138f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:09.849404 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946f6ea-29b1-4f0f-a62a-5e90eaed138f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:09.849785 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946f6ea-29b1-4f0f-a62a-5e90eaed138f None admin] PATH_INFO: `/v3/users/a618269d48c34c1da396412788af1a3d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:09.852082 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b946f6ea-29b1-4f0f-a62a-5e90eaed138f None admin] RBAC: Authorizing `identity:get_user(user_id=a618269d48c34c1da396412788af1a3d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:09.854810 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b946f6ea-29b1-4f0f-a62a-5e90eaed138f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:09.858715 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 112/227] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:17:09 2022] GET /identity/v3/users/a618269d48c34c1da396412788af1a3d => generated 283 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:09.860591 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946f6ea-29b1-4f0f-a62a-5e90eaed138f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:09.937893 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755d673-4578-4f5a-8726-2c31c4f8f629 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:10.000076 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc48098-cb1e-44db-ad07-0566f5e8949f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:10.001230 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc48098-cb1e-44db-ad07-0566f5e8949f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:10.001712 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc48098-cb1e-44db-ad07-0566f5e8949f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:10.002917 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc48098-cb1e-44db-ad07-0566f5e8949f None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:10.004277 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc48098-cb1e-44db-ad07-0566f5e8949f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:10.005007 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc48098-cb1e-44db-ad07-0566f5e8949f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:10.007766 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc48098-cb1e-44db-ad07-0566f5e8949f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:10.013093 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:10.030708 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 116/228] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:17:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:10.037765 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc48098-cb1e-44db-ad07-0566f5e8949f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:10.065817 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946f6ea-29b1-4f0f-a62a-5e90eaed138f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:10.111038 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:10.125816 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fccf59a-6e06-4200-9a02-d770503bb29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:10.126695 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fccf59a-6e06-4200-9a02-d770503bb29a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:10.126978 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fccf59a-6e06-4200-9a02-d770503bb29a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:10.127258 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fccf59a-6e06-4200-9a02-d770503bb29a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:10.128551 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fccf59a-6e06-4200-9a02-d770503bb29a None admin] RBAC: Adding query filter params (role.id=70699fb0eae7441999835fa3b50350b3, scope.project.id=1264ceb6aa114fbf81b3d48cd8874c8f, user.id=ddf67c24de3243b2b6cdb4db14ed1ad5) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:10.129423 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fccf59a-6e06-4200-9a02-d770503bb29a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:10.131334 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fccf59a-6e06-4200-9a02-d770503bb29a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:10.141135 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 113/229] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:17:09 2022] GET /identity/v3/role_assignments?user.id=ddf67c24de3243b2b6cdb4db14ed1ad5&scope.project.id=1264ceb6aa114fbf81b3d48cd8874c8f&role.id=70699fb0eae7441999835fa3b50350b3 => generated 259 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:10.141917 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fccf59a-6e06-4200-9a02-d770503bb29a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:10.227197 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc48098-cb1e-44db-ad07-0566f5e8949f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:10.303287 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c0d43-6c20-4e59-bff2-6be6266c319d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:10.304146 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c0d43-6c20-4e59-bff2-6be6266c319d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:10.304431 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c0d43-6c20-4e59-bff2-6be6266c319d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:10.304708 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c0d43-6c20-4e59-bff2-6be6266c319d None admin] PATH_INFO: `/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:10.306289 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086c0d43-6c20-4e59-bff2-6be6266c319d None admin] RBAC: Authorizing `identity:get_project(project_id=1264ceb6aa114fbf81b3d48cd8874c8f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:10.310301 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086c0d43-6c20-4e59-bff2-6be6266c319d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:10.310680 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 117/230] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:17:10 2022] GET /identity/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f => generated 302 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:10.318118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c0d43-6c20-4e59-bff2-6be6266c319d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:10.415584 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:10.448879 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fccf59a-6e06-4200-9a02-d770503bb29a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:10.476099 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c0d43-6c20-4e59-bff2-6be6266c319d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:10.509340 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6556b1b3-66af-40a9-8fb4-53458e4e5ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:10.510275 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6556b1b3-66af-40a9-8fb4-53458e4e5ebe None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:10.510574 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6556b1b3-66af-40a9-8fb4-53458e4e5ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:10.510913 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6556b1b3-66af-40a9-8fb4-53458e4e5ebe None admin] PATH_INFO: `/v3/users/a618269d48c34c1da396412788af1a3d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:10.512192 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6556b1b3-66af-40a9-8fb4-53458e4e5ebe None admin] RBAC: Authorizing `identity:get_user(user_id=a618269d48c34c1da396412788af1a3d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:10.513522 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6556b1b3-66af-40a9-8fb4-53458e4e5ebe None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:10.515212 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 114/231] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:17:10 2022] GET /identity/v3/users/a618269d48c34c1da396412788af1a3d => generated 283 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:10.520989 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6556b1b3-66af-40a9-8fb4-53458e4e5ebe None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:10.522025 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f10016-60ed-42bc-aa24-3e847879f3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:10.523191 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f10016-60ed-42bc-aa24-3e847879f3fa None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:10.523473 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f10016-60ed-42bc-aa24-3e847879f3fa None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:10.523748 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f10016-60ed-42bc-aa24-3e847879f3fa None admin] PATH_INFO: `/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f/users/a618269d48c34c1da396412788af1a3d/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:10.526215 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f10016-60ed-42bc-aa24-3e847879f3fa None admin] RBAC: Authorizing `identity:create_grant(project_id=1264ceb6aa114fbf81b3d48cd8874c8f, user_id=a618269d48c34c1da396412788af1a3d, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:10.527806 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f10016-60ed-42bc-aa24-3e847879f3fa None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:10.550425 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 118/232] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 21:17:10 2022] PUT /identity/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f/users/a618269d48c34c1da396412788af1a3d/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:17:10.601555 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:10.655498 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6556b1b3-66af-40a9-8fb4-53458e4e5ebe None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:10.709256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1371546-c614-49ba-b306-07a674474e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:10.710209 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1371546-c614-49ba-b306-07a674474e5f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:10.710570 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1371546-c614-49ba-b306-07a674474e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:10.710850 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1371546-c614-49ba-b306-07a674474e5f None admin] PATH_INFO: `/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:10.715216 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1371546-c614-49ba-b306-07a674474e5f None admin] RBAC: Authorizing `identity:get_project(project_id=1264ceb6aa114fbf81b3d48cd8874c8f)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:10.716792 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1371546-c614-49ba-b306-07a674474e5f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:10.719119 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 115/233] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:17:10 2022] GET /identity/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:17:10.725179 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f10016-60ed-42bc-aa24-3e847879f3fa None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:10.851789 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f10016-60ed-42bc-aa24-3e847879f3fa None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:10.882894 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:10.894756 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c12dbe-68c3-44d8-84ba-367824d35b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:10.895718 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c12dbe-68c3-44d8-84ba-367824d35b9b None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:10.896009 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c12dbe-68c3-44d8-84ba-367824d35b9b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:10.896291 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c12dbe-68c3-44d8-84ba-367824d35b9b None admin] PATH_INFO: `/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f/users/a618269d48c34c1da396412788af1a3d/roles/566c421659204df185f8b77bf3bb3afc` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:10.899734 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c12dbe-68c3-44d8-84ba-367824d35b9b None admin] RBAC: Authorizing `identity:create_grant(project_id=1264ceb6aa114fbf81b3d48cd8874c8f, user_id=a618269d48c34c1da396412788af1a3d, role_id=566c421659204df185f8b77bf3bb3afc)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:10.903029 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c12dbe-68c3-44d8-84ba-367824d35b9b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:10.928957 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 119/234] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 21:17:10 2022] PUT /identity/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f/users/a618269d48c34c1da396412788af1a3d/roles/566c421659204df185f8b77bf3bb3afc => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:17:11.614888 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32315180-6717-4c17-933b-11430a87d054 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:11.615448 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32315180-6717-4c17-933b-11430a87d054 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:11.615448 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32315180-6717-4c17-933b-11430a87d054 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:11.617270 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 116/235] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:11.624313 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedac10-87c7-489f-8335-c882c9ebeb19 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:11.624860 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedac10-87c7-489f-8335-c882c9ebeb19 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:11.624860 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedac10-87c7-489f-8335-c882c9ebeb19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:11.651404 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-7eedac10-87c7-489f-8335-c882c9ebeb19 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:11.652640 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7eedac10-87c7-489f-8335-c882c9ebeb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:11.658639 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e3585-70d1-404a-8bc3-0052f75d7112 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:11.658948 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e3585-70d1-404a-8bc3-0052f75d7112 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:11.659208 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e3585-70d1-404a-8bc3-0052f75d7112 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:11.660959 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 117/236] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:11.667115 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc1f238-d942-4b01-8379-31a761435935 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:11.667452 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc1f238-d942-4b01-8379-31a761435935 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:11.667691 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc1f238-d942-4b01-8379-31a761435935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:11.697270 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-bdc1f238-d942-4b01-8379-31a761435935 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:11.698515 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-bdc1f238-d942-4b01-8379-31a761435935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:11.796396 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 120/237] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:17:11.822499 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eedac10-87c7-489f-8335-c882c9ebeb19 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:11.823589 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 118/238] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:17:11.859631 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc1f238-d942-4b01-8379-31a761435935 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:11.975901 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:12.005342 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eedac10-87c7-489f-8335-c882c9ebeb19 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:12.026780 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:12.052270 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c386615-7f04-457c-81e6-9562b36ca7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:12.053155 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c386615-7f04-457c-81e6-9562b36ca7c4 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:12.053448 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c386615-7f04-457c-81e6-9562b36ca7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:12.053786 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c386615-7f04-457c-81e6-9562b36ca7c4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:12.068690 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c386615-7f04-457c-81e6-9562b36ca7c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:12.077336 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-0c386615-7f04-457c-81e6-9562b36ca7c4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 21:17:12.077336 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:12.077336 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:12.077336 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:12.077336 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:12.077336 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:12.077336 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:12.077336 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:12.077336 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:12.077336 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:12.077336 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:12.077336 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:12.077336 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:12.077336 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:12.077336 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:12.077336 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:12.078337 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:12.078337 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:12.078337 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:12.078337 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:12.078337 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:12.078337 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:12.078337 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:12.078337 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:12.078337 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:12.078337 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:12.078337 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:12.078337 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:12.078337 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:12.078337 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:12.078337 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:12.078337 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:12.079173 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:12.079173 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:12.079173 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:12.079173 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:12.079173 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:12.079173 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:12.079173 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:12.079173 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:12.079173 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:12.079173 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:12.079173 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 21:17:12.079173 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:12.079173 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 121/239] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:17:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:12.085145 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c386615-7f04-457c-81e6-9562b36ca7c4 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:12.096578 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc1f238-d942-4b01-8379-31a761435935 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:12.196103 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5275e3f-0296-4fb8-84fe-6e17c09d6a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:12.197298 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5275e3f-0296-4fb8-84fe-6e17c09d6a0a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:12.197718 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5275e3f-0296-4fb8-84fe-6e17c09d6a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:12.198115 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5275e3f-0296-4fb8-84fe-6e17c09d6a0a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:12.203126 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5275e3f-0296-4fb8-84fe-6e17c09d6a0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:12.204772 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5275e3f-0296-4fb8-84fe-6e17c09d6a0a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:12.206170 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c386615-7f04-457c-81e6-9562b36ca7c4 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:12.206756 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 119/240] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:17:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:12.214641 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5275e3f-0296-4fb8-84fe-6e17c09d6a0a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:12.273146 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b8235-b9cd-4b18-b9e0-4e00b94c8608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:12.273752 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b8235-b9cd-4b18-b9e0-4e00b94c8608 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:12.274048 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b8235-b9cd-4b18-b9e0-4e00b94c8608 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:12.274382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b8235-b9cd-4b18-b9e0-4e00b94c8608 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:12.278449 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0b8235-b9cd-4b18-b9e0-4e00b94c8608 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:12.278769 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0b8235-b9cd-4b18-b9e0-4e00b94c8608 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:12.280182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0b8235-b9cd-4b18-b9e0-4e00b94c8608 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:12.301696 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 122/241] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 21:17:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:12.307003 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b8235-b9cd-4b18-b9e0-4e00b94c8608 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:12.386614 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:12.428582 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5275e3f-0296-4fb8-84fe-6e17c09d6a0a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:12.474781 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:12.485221 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847021b5-75bf-4c1b-b837-048929ff8e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:12.486344 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847021b5-75bf-4c1b-b837-048929ff8e06 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:12.486842 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847021b5-75bf-4c1b-b837-048929ff8e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:12.487261 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847021b5-75bf-4c1b-b837-048929ff8e06 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:12.488208 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847021b5-75bf-4c1b-b837-048929ff8e06 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:12.489810 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847021b5-75bf-4c1b-b837-048929ff8e06 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:12.497079 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b8235-b9cd-4b18-b9e0-4e00b94c8608 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:12.518813 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 120/242] 10.222.0.65 () {60 vars in 1132 bytes} [Thu May 19 21:17:12 2022] POST /identity/v3/users => generated 293 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:17:12.556384 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51870f9-c289-4de4-8a25-f287ae7581f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:12.557585 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51870f9-c289-4de4-8a25-f287ae7581f4 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:12.558007 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51870f9-c289-4de4-8a25-f287ae7581f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:12.558449 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51870f9-c289-4de4-8a25-f287ae7581f4 None admin] PATH_INFO: `/v3/users/a0cb3047d64e4c64926ae71f71d6c667` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:12.577796 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51870f9-c289-4de4-8a25-f287ae7581f4 None admin] RBAC: Authorizing `identity:get_user(user_id=a0cb3047d64e4c64926ae71f71d6c667)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:12.579562 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51870f9-c289-4de4-8a25-f287ae7581f4 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:12.581922 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 123/243] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:17:12 2022] GET /identity/v3/users/a0cb3047d64e4c64926ae71f71d6c667 => generated 297 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:17:12.591560 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847021b5-75bf-4c1b-b837-048929ff8e06 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:12.712090 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847021b5-75bf-4c1b-b837-048929ff8e06 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:12.756794 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358957ca-9c3c-4623-94ed-774abdabcb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:12.757464 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358957ca-9c3c-4623-94ed-774abdabcb03 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:12.757747 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358957ca-9c3c-4623-94ed-774abdabcb03 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:12.758019 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358957ca-9c3c-4623-94ed-774abdabcb03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:12.820509 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:12.840081 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358957ca-9c3c-4623-94ed-774abdabcb03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:12.848711 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358957ca-9c3c-4623-94ed-774abdabcb03 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:12.962171 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 121/244] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:17:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:17:12.968329 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51870f9-c289-4de4-8a25-f287ae7581f4 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:13.049480 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e1119-03a3-42ff-a4f2-780588769269 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:13.050108 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e1119-03a3-42ff-a4f2-780588769269 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:13.050619 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e1119-03a3-42ff-a4f2-780588769269 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:13.052802 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 122/245] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:17:13.058703 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d2b74-8412-46e8-b151-098f13fa4df9 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:13.059343 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d2b74-8412-46e8-b151-098f13fa4df9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:13.059863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d2b74-8412-46e8-b151-098f13fa4df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:13.078983 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-4f2d2b74-8412-46e8-b151-098f13fa4df9 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:13.080516 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4f2d2b74-8412-46e8-b151-098f13fa4df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:13.091657 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:13.099943 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51870f9-c289-4de4-8a25-f287ae7581f4 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:13.161801 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5666fb-0c05-46d4-94ae-4a37e5b2b84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:13.163397 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5666fb-0c05-46d4-94ae-4a37e5b2b84e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:13.165206 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5666fb-0c05-46d4-94ae-4a37e5b2b84e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:13.165691 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5666fb-0c05-46d4-94ae-4a37e5b2b84e None admin] PATH_INFO: `/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:13.168077 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5666fb-0c05-46d4-94ae-4a37e5b2b84e None admin] RBAC: Authorizing `identity:get_project(project_id=1264ceb6aa114fbf81b3d48cd8874c8f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:13.171300 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5666fb-0c05-46d4-94ae-4a37e5b2b84e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:13.173094 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 124/246] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:17:12 2022] GET /identity/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:13.173300 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 123/247] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:17:13.178900 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5666fb-0c05-46d4-94ae-4a37e5b2b84e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:13.198104 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d2b74-8412-46e8-b151-098f13fa4df9 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:13.219073 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:13.325650 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5666fb-0c05-46d4-94ae-4a37e5b2b84e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:13.326419 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d2b74-8412-46e8-b151-098f13fa4df9 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:13.368769 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12177596-bdda-4e09-a963-648b9a4457d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:13.369869 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12177596-bdda-4e09-a963-648b9a4457d5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:13.370344 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12177596-bdda-4e09-a963-648b9a4457d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:13.370829 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12177596-bdda-4e09-a963-648b9a4457d5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:13.373388 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb48af46-b171-434d-a035-e44ab8ce51a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:13.374068 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48af46-b171-434d-a035-e44ab8ce51a3 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:13.374453 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48af46-b171-434d-a035-e44ab8ce51a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:13.375167 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48af46-b171-434d-a035-e44ab8ce51a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:13.376025 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb48af46-b171-434d-a035-e44ab8ce51a3 None admin] RBAC: Adding query filter params (role.id=34a7a66da3654831a115d77e40a989da, scope.project.id=1264ceb6aa114fbf81b3d48cd8874c8f, user.id=a0cb3047d64e4c64926ae71f71d6c667) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:13.376335 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb48af46-b171-434d-a035-e44ab8ce51a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:13.379291 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb48af46-b171-434d-a035-e44ab8ce51a3 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:13.385858 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 125/248] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:17:13 2022] GET /identity/v3/role_assignments?user.id=a0cb3047d64e4c64926ae71f71d6c667&scope.project.id=1264ceb6aa114fbf81b3d48cd8874c8f&role.id=34a7a66da3654831a115d77e40a989da => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:17:13.387878 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12177596-bdda-4e09-a963-648b9a4457d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:13.396124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-12177596-bdda-4e09-a963-648b9a4457d5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 21:17:13.396124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:13.396124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:13.396124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:13.396124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:13.396124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:13.396124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:13.396124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:13.396124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:13.396124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:13.396124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:13.396124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:13.396124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:13.396124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:13.396124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:13.396124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:13.397204 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:13.397204 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:13.397204 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:13.397204 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:13.397204 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:13.397204 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:13.397204 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:13.397204 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:13.397204 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:13.397204 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:13.397204 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:13.397204 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:17:13.397204 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:13.397204 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:13.397204 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:13.397204 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:13.397985 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:13.397985 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:13.397985 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:13.397985 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:13.397985 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:13.397985 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:13.397985 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:13.397985 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:13.397985 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:13.397985 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:13.397985 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 21:17:13.397985 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:17:13.397985 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 124/249] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:17:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:17:13.404779 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb48af46-b171-434d-a035-e44ab8ce51a3 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:13.437311 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:13.494030 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5ace3-acd9-4939-8a54-e7a4a66e3634 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:13.494699 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5ace3-acd9-4939-8a54-e7a4a66e3634 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:13.495200 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5ace3-acd9-4939-8a54-e7a4a66e3634 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:13.497248 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 125/250] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:13.508116 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f48838-a338-4a58-970b-e8b04edd8915 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:13.508829 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f48838-a338-4a58-970b-e8b04edd8915 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:13.509450 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f48838-a338-4a58-970b-e8b04edd8915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:13.532437 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-21f48838-a338-4a58-970b-e8b04edd8915 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:13.537158 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-21f48838-a338-4a58-970b-e8b04edd8915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:13.546503 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb48af46-b171-434d-a035-e44ab8ce51a3 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:13.651212 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9e40fc-8a78-40bf-9893-8bbc1b0d6dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:13.651840 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9e40fc-8a78-40bf-9893-8bbc1b0d6dcc None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:13.652181 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9e40fc-8a78-40bf-9893-8bbc1b0d6dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:13.652428 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9e40fc-8a78-40bf-9893-8bbc1b0d6dcc None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:13.653205 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9e40fc-8a78-40bf-9893-8bbc1b0d6dcc None admin] RBAC: Adding query filter params (name=member) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:13.653513 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9e40fc-8a78-40bf-9893-8bbc1b0d6dcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:13.654957 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9e40fc-8a78-40bf-9893-8bbc1b0d6dcc None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:13.669524 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 126/251] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 19 21:17:13.676641 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 126/252] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 21:17:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:13.682751 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f48838-a338-4a58-970b-e8b04edd8915 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:13.686498 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:13.710232 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9e40fc-8a78-40bf-9893-8bbc1b0d6dcc None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:13.800655 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f48838-a338-4a58-970b-e8b04edd8915 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:13.845683 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d772051b-ea47-4c1d-8655-cc62a531c36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:13.845891 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d772051b-ea47-4c1d-8655-cc62a531c36b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:13.846151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d772051b-ea47-4c1d-8655-cc62a531c36b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:13.846343 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d772051b-ea47-4c1d-8655-cc62a531c36b None admin] PATH_INFO: `/v3/users/ddf67c24de3243b2b6cdb4db14ed1ad5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:13.851252 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d772051b-ea47-4c1d-8655-cc62a531c36b None admin] RBAC: Authorizing `identity:get_user(user_id=ddf67c24de3243b2b6cdb4db14ed1ad5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:13.853571 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d772051b-ea47-4c1d-8655-cc62a531c36b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:13.859861 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 127/253] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:17:13 2022] GET /identity/v3/users/ddf67c24de3243b2b6cdb4db14ed1ad5 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:13.868557 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d772051b-ea47-4c1d-8655-cc62a531c36b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:13.954076 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:13.973258 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9e40fc-8a78-40bf-9893-8bbc1b0d6dcc None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:14.016549 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:14.036079 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a4585-53ac-42ab-be74-31a45f3fd34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:14.036918 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a4585-53ac-42ab-be74-31a45f3fd34d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:14.037205 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a4585-53ac-42ab-be74-31a45f3fd34d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:14.037483 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a4585-53ac-42ab-be74-31a45f3fd34d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:14.051538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d772051b-ea47-4c1d-8655-cc62a531c36b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:14.060052 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a4585-53ac-42ab-be74-31a45f3fd34d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:14.076343 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-4c9a4585-53ac-42ab-be74-31a45f3fd34d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:17:14.076343 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:14.076343 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:14.076343 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:14.076343 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:14.076343 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:14.076343 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:14.076343 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:14.076343 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:14.076343 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:14.076343 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:14.076343 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:14.076343 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:14.076343 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:14.076343 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:14.076343 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:14.077348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:14.077348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:14.077348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:14.077348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:14.077348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:14.077348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:14.077348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:14.077348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:14.077348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:14.077348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:14.077348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:14.077348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:14.077348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:14.077348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:14.077348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:14.077348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:14.078160 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:14.078160 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:14.078160 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:14.078160 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:14.078160 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:14.078160 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:14.078160 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:14.078160 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:14.078160 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:14.078160 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:14.078160 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:17:14.078160 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:14.078160 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 127/254] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 21:17:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:17:14.091202 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b3b434-6dc6-4e83-89ca-a8ff2548b994 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:14.091536 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b3b434-6dc6-4e83-89ca-a8ff2548b994 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:14.091770 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b3b434-6dc6-4e83-89ca-a8ff2548b994 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:14.093365 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 128/255] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:14 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:14.093971 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b3b434-6dc6-4e83-89ca-a8ff2548b994 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:14.095328 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307eb808-eb55-430e-95d4-1e4caed084d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:14.096562 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307eb808-eb55-430e-95d4-1e4caed084d2 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:14.097040 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307eb808-eb55-430e-95d4-1e4caed084d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:14.097506 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307eb808-eb55-430e-95d4-1e4caed084d2 None admin] PATH_INFO: `/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:14.099183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307eb808-eb55-430e-95d4-1e4caed084d2 None admin] RBAC: Authorizing `identity:get_project(project_id=1264ceb6aa114fbf81b3d48cd8874c8f)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:14.100927 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307eb808-eb55-430e-95d4-1e4caed084d2 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:14.102952 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 128/256] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:17:13 2022] GET /identity/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:17:14.104428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae451c-0820-40b3-af92-df32411974d1 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:14.104919 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae451c-0820-40b3-af92-df32411974d1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:14.105528 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae451c-0820-40b3-af92-df32411974d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:14.124749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-14ae451c-0820-40b3-af92-df32411974d1 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:14.134549 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-14ae451c-0820-40b3-af92-df32411974d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:14.238267 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b3b434-6dc6-4e83-89ca-a8ff2548b994 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:14.301818 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554aa8e-30c8-4826-9eca-c79124394045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:14.303639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554aa8e-30c8-4826-9eca-c79124394045 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:14.303930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554aa8e-30c8-4826-9eca-c79124394045 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:14.304212 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554aa8e-30c8-4826-9eca-c79124394045 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:14.304973 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9554aa8e-30c8-4826-9eca-c79124394045 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:14.305286 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9554aa8e-30c8-4826-9eca-c79124394045 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:14.306798 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9554aa8e-30c8-4826-9eca-c79124394045 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:14.318901 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 129/257] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:17:14.319797 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae451c-0820-40b3-af92-df32411974d1 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:14.334087 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 129/258] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:17:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:14.343283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554aa8e-30c8-4826-9eca-c79124394045 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:14.387970 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:14.464946 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:14.494618 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554aa8e-30c8-4826-9eca-c79124394045 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:14.532394 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae451c-0820-40b3-af92-df32411974d1 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:14.534024 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771c6679-bac6-435f-be5e-3b42c49b36c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:14.534999 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c6679-bac6-435f-be5e-3b42c49b36c8 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:14.535273 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c6679-bac6-435f-be5e-3b42c49b36c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:14.535542 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c6679-bac6-435f-be5e-3b42c49b36c8 None admin] PATH_INFO: `/v3/users/a618269d48c34c1da396412788af1a3d` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:14.536793 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771c6679-bac6-435f-be5e-3b42c49b36c8 None admin] RBAC: Authorizing `identity:get_user(user_id=a618269d48c34c1da396412788af1a3d)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:14.538274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771c6679-bac6-435f-be5e-3b42c49b36c8 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:14.541438 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 130/259] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:17:14 2022] GET /identity/v3/users/a618269d48c34c1da396412788af1a3d => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:14.542395 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771c6679-bac6-435f-be5e-3b42c49b36c8 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:14.629764 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd7666-e1d6-4fb8-a326-ab92d9aaeb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:14.630719 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd7666-e1d6-4fb8-a326-ab92d9aaeb87 None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:14.630995 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd7666-e1d6-4fb8-a326-ab92d9aaeb87 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:14.631264 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd7666-e1d6-4fb8-a326-ab92d9aaeb87 None admin] PATH_INFO: `/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f/users/ddf67c24de3243b2b6cdb4db14ed1ad5/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:14.636408 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cd7666-e1d6-4fb8-a326-ab92d9aaeb87 None admin] RBAC: Authorizing `identity:create_grant(project_id=1264ceb6aa114fbf81b3d48cd8874c8f, user_id=ddf67c24de3243b2b6cdb4db14ed1ad5, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:14.639934 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cd7666-e1d6-4fb8-a326-ab92d9aaeb87 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:14.657266 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:14.669309 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 130/260] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 21:17:14 2022] PUT /identity/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f/users/ddf67c24de3243b2b6cdb4db14ed1ad5/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:17:14.670270 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd7666-e1d6-4fb8-a326-ab92d9aaeb87 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:14.701119 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771c6679-bac6-435f-be5e-3b42c49b36c8 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:14.777625 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b4c41-1123-41ef-a241-5befda3583fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:14.778409 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b4c41-1123-41ef-a241-5befda3583fe None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:14.778688 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b4c41-1123-41ef-a241-5befda3583fe None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:14.778960 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b4c41-1123-41ef-a241-5befda3583fe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:14.801424 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b4c41-1123-41ef-a241-5befda3583fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:14.814760 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-4c7b4c41-1123-41ef-a241-5befda3583fe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:17:14.814760 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:14.814760 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:14.814760 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:14.814760 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:14.814760 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:14.814760 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:14.814760 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:14.814760 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:14.814760 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:14.814760 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:14.814760 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:14.814760 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:14.814760 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:14.814760 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:14.814760 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:14.815735 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:14.815735 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:14.815735 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:14.815735 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:14.815735 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:14.815735 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:14.815735 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:14.815735 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:14.815735 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:14.815735 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:14.815735 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:14.815735 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:14.815735 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:14.815735 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:14.815735 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:14.815735 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:14.816538 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:14.816538 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:14.816538 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:14.816538 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:14.816538 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:14.816538 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:14.816538 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:14.816538 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:14.816538 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:14.816538 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:14.816538 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:17:14.816538 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:14.816538 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 131/261] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 21:17:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:14.823712 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b4c41-1123-41ef-a241-5befda3583fe None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:14.906014 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd7666-e1d6-4fb8-a326-ab92d9aaeb87 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:14.971109 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:14.986390 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884bc2df-2a8f-40de-a222-72d4041cec01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:14.987064 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884bc2df-2a8f-40de-a222-72d4041cec01 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:14.987351 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884bc2df-2a8f-40de-a222-72d4041cec01 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:14.987666 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884bc2df-2a8f-40de-a222-72d4041cec01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:15.039356 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b4c41-1123-41ef-a241-5befda3583fe None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:15.113651 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee2d7e8-d4b1-44f9-86e7-85140a300960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:15.114365 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2d7e8-d4b1-44f9-86e7-85140a300960 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:15.114652 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2d7e8-d4b1-44f9-86e7-85140a300960 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:15.114928 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2d7e8-d4b1-44f9-86e7-85140a300960 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:15.115680 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee2d7e8-d4b1-44f9-86e7-85140a300960 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:15.115987 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee2d7e8-d4b1-44f9-86e7-85140a300960 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:15.117389 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee2d7e8-d4b1-44f9-86e7-85140a300960 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:15.126078 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884bc2df-2a8f-40de-a222-72d4041cec01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:15.127611 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884bc2df-2a8f-40de-a222-72d4041cec01 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:15.147514 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 132/262] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:17:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:15.153882 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee2d7e8-d4b1-44f9-86e7-85140a300960 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:15.157460 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:15.360314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 131/263] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:17:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 690 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:17:15.366011 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884bc2df-2a8f-40de-a222-72d4041cec01 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:15.381187 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee2d7e8-d4b1-44f9-86e7-85140a300960 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:15.398127 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:15.463788 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f264b2b-64ef-4b92-aefe-00ca940e5ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:15.464695 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f264b2b-64ef-4b92-aefe-00ca940e5ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:15.464974 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f264b2b-64ef-4b92-aefe-00ca940e5ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:15.465249 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f264b2b-64ef-4b92-aefe-00ca940e5ae8 None admin] PATH_INFO: `/v3/users/a618269d48c34c1da396412788af1a3d` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:15.471030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f264b2b-64ef-4b92-aefe-00ca940e5ae8 None admin] RBAC: Authorizing `identity:get_user(user_id=a618269d48c34c1da396412788af1a3d)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:15.472513 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f264b2b-64ef-4b92-aefe-00ca940e5ae8 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:15.474605 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 133/264] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:17:15 2022] GET /identity/v3/users/a618269d48c34c1da396412788af1a3d => generated 283 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:15.475628 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884bc2df-2a8f-40de-a222-72d4041cec01 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:15.480334 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f264b2b-64ef-4b92-aefe-00ca940e5ae8 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:15.512667 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3487979b-3919-4e86-bc25-327ded43cdec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:15.513598 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487979b-3919-4e86-bc25-327ded43cdec None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:15.513879 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487979b-3919-4e86-bc25-327ded43cdec None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:15.514151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487979b-3919-4e86-bc25-327ded43cdec None admin] PATH_INFO: `/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:15.515695 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3487979b-3919-4e86-bc25-327ded43cdec None admin] RBAC: Authorizing `identity:get_project(project_id=1264ceb6aa114fbf81b3d48cd8874c8f)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:15.517948 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3487979b-3919-4e86-bc25-327ded43cdec None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:15.524832 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 132/265] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:17:15 2022] GET /identity/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:15.528959 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3487979b-3919-4e86-bc25-327ded43cdec None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:15.634990 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:15.639986 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3487979b-3919-4e86-bc25-327ded43cdec None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:15.664563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:15.680895 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac8341-dce2-415e-aee5-d697ffb11c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:15.681662 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac8341-dce2-415e-aee5-d697ffb11c1a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:15.681941 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac8341-dce2-415e-aee5-d697ffb11c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:15.682213 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac8341-dce2-415e-aee5-d697ffb11c1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:15.683093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ac8341-dce2-415e-aee5-d697ffb11c1a None admin] RBAC: Adding query filter params (role.id=2b714f2e5c6c469f8fa037bda1ed16f6, scope.project.id=1264ceb6aa114fbf81b3d48cd8874c8f, user.id=a618269d48c34c1da396412788af1a3d) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:15.683387 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ac8341-dce2-415e-aee5-d697ffb11c1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:15.684955 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ac8341-dce2-415e-aee5-d697ffb11c1a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:15.691621 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 133/266] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:17:15 2022] GET /identity/v3/role_assignments?user.id=a618269d48c34c1da396412788af1a3d&scope.project.id=1264ceb6aa114fbf81b3d48cd8874c8f&role.id=2b714f2e5c6c469f8fa037bda1ed16f6 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:17:15.723947 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f264b2b-64ef-4b92-aefe-00ca940e5ae8 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:15.761656 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2394f63-0054-407c-b9a0-7dc2613b2631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:15.762363 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2394f63-0054-407c-b9a0-7dc2613b2631 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:15.762831 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2394f63-0054-407c-b9a0-7dc2613b2631 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:15.763118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2394f63-0054-407c-b9a0-7dc2613b2631 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:15.873441 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2394f63-0054-407c-b9a0-7dc2613b2631 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:15.875284 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2394f63-0054-407c-b9a0-7dc2613b2631 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:15.989121 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:16.025546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 134/267] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:17:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 21:17:16.031117 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac8341-dce2-415e-aee5-d697ffb11c1a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:16.154916 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac8341-dce2-415e-aee5-d697ffb11c1a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:16.163973 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77057e-f0c7-441f-b220-bce485922780 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:16.164502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77057e-f0c7-441f-b220-bce485922780 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:16.164502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77057e-f0c7-441f-b220-bce485922780 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:16.166624 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 135/268] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:17:16.176411 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73698d9a-cb12-4baa-a552-c6a3295b9353 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:16.176720 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73698d9a-cb12-4baa-a552-c6a3295b9353 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:16.176960 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73698d9a-cb12-4baa-a552-c6a3295b9353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:16.203475 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-73698d9a-cb12-4baa-a552-c6a3295b9353 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:16.204756 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-73698d9a-cb12-4baa-a552-c6a3295b9353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:16.240024 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0aa47-1909-4bdd-8082-0eb3fd6ae240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:16.240893 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0aa47-1909-4bdd-8082-0eb3fd6ae240 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:16.241157 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0aa47-1909-4bdd-8082-0eb3fd6ae240 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:16.241426 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0aa47-1909-4bdd-8082-0eb3fd6ae240 None admin] PATH_INFO: `/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:16.244235 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc0aa47-1909-4bdd-8082-0eb3fd6ae240 None admin] RBAC: Authorizing `identity:get_project(project_id=1264ceb6aa114fbf81b3d48cd8874c8f)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:16.245725 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc0aa47-1909-4bdd-8082-0eb3fd6ae240 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:16.249895 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 134/269] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:17:16 2022] GET /identity/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:16.254621 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0aa47-1909-4bdd-8082-0eb3fd6ae240 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:16.309787 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:16.331391 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 136/270] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:17:16.393073 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73698d9a-cb12-4baa-a552-c6a3295b9353 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:16.402005 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0aa47-1909-4bdd-8082-0eb3fd6ae240 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:16.460374 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34696f9-60df-4505-ae27-9713794365e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:16.461397 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34696f9-60df-4505-ae27-9713794365e2 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:16.461950 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34696f9-60df-4505-ae27-9713794365e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:16.462499 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34696f9-60df-4505-ae27-9713794365e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:16.463847 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34696f9-60df-4505-ae27-9713794365e2 None admin] RBAC: Adding query filter params (role.id=566c421659204df185f8b77bf3bb3afc, scope.project.id=1264ceb6aa114fbf81b3d48cd8874c8f, user.id=a618269d48c34c1da396412788af1a3d) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:16.464270 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34696f9-60df-4505-ae27-9713794365e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:16.465832 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34696f9-60df-4505-ae27-9713794365e2 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:16.473019 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 135/271] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:17:16 2022] GET /identity/v3/role_assignments?user.id=a618269d48c34c1da396412788af1a3d&scope.project.id=1264ceb6aa114fbf81b3d48cd8874c8f&role.id=566c421659204df185f8b77bf3bb3afc => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:16.474226 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0885da52-7b6d-420b-b45e-823e3b99f598 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:16.474616 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0885da52-7b6d-420b-b45e-823e3b99f598 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:16.474967 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0885da52-7b6d-420b-b45e-823e3b99f598 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:16.476791 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 136/272] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:16.482792 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7993f41-413e-4e28-8ec3-7eeb910d2b0b None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:16.483342 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7993f41-413e-4e28-8ec3-7eeb910d2b0b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:16.483822 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7993f41-413e-4e28-8ec3-7eeb910d2b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:16.502982 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b7993f41-413e-4e28-8ec3-7eeb910d2b0b None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:16.504463 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b7993f41-413e-4e28-8ec3-7eeb910d2b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:16.548761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:16.557121 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73698d9a-cb12-4baa-a552-c6a3295b9353 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:16.588007 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 137/273] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:17:16.609889 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f109c-1554-4dc4-a62d-dc1f5688f422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:16.611119 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f109c-1554-4dc4-a62d-dc1f5688f422 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:16.611569 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f109c-1554-4dc4-a62d-dc1f5688f422 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:16.612004 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f109c-1554-4dc4-a62d-dc1f5688f422 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:16.613574 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f109c-1554-4dc4-a62d-dc1f5688f422 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:16.615527 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f109c-1554-4dc4-a62d-dc1f5688f422 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:16.618514 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 137/274] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:17:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:17:16.623778 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7993f41-413e-4e28-8ec3-7eeb910d2b0b None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:16.635402 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f109c-1554-4dc4-a62d-dc1f5688f422 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:16.672873 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:16.785342 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7993f41-413e-4e28-8ec3-7eeb910d2b0b None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:16.797153 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f109c-1554-4dc4-a62d-dc1f5688f422 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:16.836342 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b7d8cc-fda5-400d-978a-4e9febaf3203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:16.838382 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7d8cc-fda5-400d-978a-4e9febaf3203 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:16.839200 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7d8cc-fda5-400d-978a-4e9febaf3203 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:16.839691 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7d8cc-fda5-400d-978a-4e9febaf3203 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:16.841916 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b7d8cc-fda5-400d-978a-4e9febaf3203 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:16.845563 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b7d8cc-fda5-400d-978a-4e9febaf3203 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:16.857997 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2204396f-0cc2-4c86-b0f8-38e9a4032f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:16.861823 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204396f-0cc2-4c86-b0f8-38e9a4032f20 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:16.862153 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204396f-0cc2-4c86-b0f8-38e9a4032f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:16.862494 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204396f-0cc2-4c86-b0f8-38e9a4032f20 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:16.885630 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 138/275] 10.222.0.65 () {60 vars in 1132 bytes} [Thu May 19 21:17:16 2022] POST /identity/v3/users => generated 293 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:17:16.887651 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde9e22-1594-4d93-8678-e923ccbdc5a6 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:16.888071 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde9e22-1594-4d93-8678-e923ccbdc5a6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:16.888518 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde9e22-1594-4d93-8678-e923ccbdc5a6 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:16.889776 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2204396f-0cc2-4c86-b0f8-38e9a4032f20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:16.897280 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 139/276] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:16.899118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-2204396f-0cc2-4c86-b0f8-38e9a4032f20 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 21:17:16.899118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:16.899118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:16.899118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:16.899118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:16.899118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:16.899118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:16.899118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:16.899118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:16.899118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:16.899118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:16.899118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:16.899118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:16.899118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:16.899118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:16.899118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:16.900332 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:16.900332 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:16.900332 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:16.900332 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:16.900332 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:16.900332 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:16.900332 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:16.900332 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:16.900332 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:16.900332 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:16.900332 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:16.900332 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:16.900332 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:16.900332 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:16.900332 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:16.900332 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:16.901174 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:16.901174 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:16.901174 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:16.901174 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:16.901174 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:16.901174 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:16.901174 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:16.901174 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:16.901174 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:16.901174 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:16.901174 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 21:17:16.901174 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:16.901850 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c1282-3f29-44c8-87e7-796260aebdd6 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:16.901850 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c1282-3f29-44c8-87e7-796260aebdd6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:16.901850 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c1282-3f29-44c8-87e7-796260aebdd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:16.905129 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 138/277] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:17:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:16.914701 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2204396f-0cc2-4c86-b0f8-38e9a4032f20 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:16.928572 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e43c1282-3f29-44c8-87e7-796260aebdd6 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:16.930610 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e43c1282-3f29-44c8-87e7-796260aebdd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:16.980824 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:17.040680 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 140/278] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:17:17.062710 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43c1282-3f29-44c8-87e7-796260aebdd6 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:17.078514 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:17.093906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2204396f-0cc2-4c86-b0f8-38e9a4032f20 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:17.138736 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f410c-06b6-40f1-9189-ed212148a5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:17.139436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f410c-06b6-40f1-9189-ed212148a5fd None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:17.139722 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f410c-06b6-40f1-9189-ed212148a5fd None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:17.140002 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f410c-06b6-40f1-9189-ed212148a5fd None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:17.140969 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f410c-06b6-40f1-9189-ed212148a5fd None admin] RBAC: Adding query filter params (name=member) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:17.141285 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f410c-06b6-40f1-9189-ed212148a5fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:17.143179 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f410c-06b6-40f1-9189-ed212148a5fd None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:17.158270 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 139/279] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 21:17:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:17.160724 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f410c-06b6-40f1-9189-ed212148a5fd None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:17.205245 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43c1282-3f29-44c8-87e7-796260aebdd6 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:17.256163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89af5a1-8d62-4448-805c-117600eebee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:17.256993 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89af5a1-8d62-4448-805c-117600eebee8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:17.257275 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89af5a1-8d62-4448-805c-117600eebee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:17.257547 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89af5a1-8d62-4448-805c-117600eebee8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:17.277177 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89af5a1-8d62-4448-805c-117600eebee8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:17.286860 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-f89af5a1-8d62-4448-805c-117600eebee8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 21:17:17.286860 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:17.286860 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:17.286860 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:17.286860 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:17.286860 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:17.286860 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:17.286860 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:17.286860 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:17.286860 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:17.286860 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:17.286860 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:17.286860 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:17.286860 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:17.286860 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:17.286860 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:17.287804 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:17.287804 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:17.287804 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:17.287804 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:17.287804 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:17.287804 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:17.287804 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:17.287804 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:17.287804 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:17.287804 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:17.287804 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:17.287804 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:17:17.287804 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:17.287804 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:17.287804 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:17.287804 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:17.288591 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:17.288591 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:17.288591 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:17.288591 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:17.288591 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:17.288591 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:17.288591 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:17.288591 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:17.288591 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:17.288591 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:17.288591 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 21:17:17.288591 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:17:17.289762 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 141/280] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:17:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:17:17.290422 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:17.295154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89af5a1-8d62-4448-805c-117600eebee8 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:17.301567 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f410c-06b6-40f1-9189-ed212148a5fd None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:17.341949 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc003af-b04c-4ba5-abb9-889195475276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:17.342881 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc003af-b04c-4ba5-abb9-889195475276 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:17.343159 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc003af-b04c-4ba5-abb9-889195475276 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:17.343537 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc003af-b04c-4ba5-abb9-889195475276 None admin] PATH_INFO: `/v3/users/cbd78a2644b14db8a3056e7d57ebf032` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:17.361579 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc003af-b04c-4ba5-abb9-889195475276 None admin] RBAC: Authorizing `identity:get_user(user_id=cbd78a2644b14db8a3056e7d57ebf032)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:17.363958 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc003af-b04c-4ba5-abb9-889195475276 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:17.365965 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 140/281] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:17:17 2022] GET /identity/v3/users/cbd78a2644b14db8a3056e7d57ebf032 => generated 293 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:17:17.373185 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc003af-b04c-4ba5-abb9-889195475276 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:17.392782 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:17.436781 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89af5a1-8d62-4448-805c-117600eebee8 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:17.486308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93709bca-9972-4f46-885c-ea4e56aa012b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:17.487546 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93709bca-9972-4f46-885c-ea4e56aa012b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:17.487947 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93709bca-9972-4f46-885c-ea4e56aa012b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:17.488232 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93709bca-9972-4f46-885c-ea4e56aa012b None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:17.489024 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93709bca-9972-4f46-885c-ea4e56aa012b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:17.489337 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93709bca-9972-4f46-885c-ea4e56aa012b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:17.489881 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc003af-b04c-4ba5-abb9-889195475276 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:17.490795 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93709bca-9972-4f46-885c-ea4e56aa012b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:17.510299 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 142/282] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 21:17:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:17.515948 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93709bca-9972-4f46-885c-ea4e56aa012b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:17.548194 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb324ce-048e-4d5d-95fb-efbbf33b444e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:17.551478 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb324ce-048e-4d5d-95fb-efbbf33b444e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:17.551785 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb324ce-048e-4d5d-95fb-efbbf33b444e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:17.552087 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb324ce-048e-4d5d-95fb-efbbf33b444e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:17.552980 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb324ce-048e-4d5d-95fb-efbbf33b444e None admin] RBAC: Adding query filter params (role.id=70699fb0eae7441999835fa3b50350b3, user.id=cbd78a2644b14db8a3056e7d57ebf032, scope.system=all) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:17.553347 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb324ce-048e-4d5d-95fb-efbbf33b444e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:17.554932 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb324ce-048e-4d5d-95fb-efbbf33b444e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:17.572134 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 141/283] 10.222.0.65 () {56 vars in 1308 bytes} [Thu May 19 21:17:17 2022] GET /identity/v3/role_assignments?user.id=cbd78a2644b14db8a3056e7d57ebf032&scope.system=all&role.id=70699fb0eae7441999835fa3b50350b3 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:17.684654 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93709bca-9972-4f46-885c-ea4e56aa012b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:17.728322 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:17.737681 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f015d0e-6f6c-4b15-9d32-181bb0f07cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:17.738630 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f015d0e-6f6c-4b15-9d32-181bb0f07cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:17.738910 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f015d0e-6f6c-4b15-9d32-181bb0f07cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:17.739180 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f015d0e-6f6c-4b15-9d32-181bb0f07cb0 None admin] PATH_INFO: `/v3/users/a0cb3047d64e4c64926ae71f71d6c667` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:17.740852 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f015d0e-6f6c-4b15-9d32-181bb0f07cb0 None admin] RBAC: Authorizing `identity:get_user(user_id=a0cb3047d64e4c64926ae71f71d6c667)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:17.742388 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f015d0e-6f6c-4b15-9d32-181bb0f07cb0 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:17.744448 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 143/284] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:17:17 2022] GET /identity/v3/users/a0cb3047d64e4c64926ae71f71d6c667 => generated 297 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:17.755399 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb324ce-048e-4d5d-95fb-efbbf33b444e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:17.781511 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:17.912450 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb324ce-048e-4d5d-95fb-efbbf33b444e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:17.934151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b997f-7fc4-4cb0-81b4-65ac637896cf None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:17.934859 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b997f-7fc4-4cb0-81b4-65ac637896cf None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:17.935368 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b997f-7fc4-4cb0-81b4-65ac637896cf None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:17.937245 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 144/285] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:17:17.942958 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8967f29-d61e-43d9-9d58-fec6838f7afc None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:17.943531 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8967f29-d61e-43d9-9d58-fec6838f7afc None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:17.944004 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8967f29-d61e-43d9-9d58-fec6838f7afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:17.967998 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86519f-145a-43ca-8549-eb28dcf970e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:17.969271 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86519f-145a-43ca-8549-eb28dcf970e4 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:17.969745 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86519f-145a-43ca-8549-eb28dcf970e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:17.970207 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86519f-145a-43ca-8549-eb28dcf970e4 None admin] PATH_INFO: `/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:17.971482 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a8967f29-d61e-43d9-9d58-fec6838f7afc None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:17.972343 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c86519f-145a-43ca-8549-eb28dcf970e4 None admin] RBAC: Authorizing `identity:get_project(project_id=1264ceb6aa114fbf81b3d48cd8874c8f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:17.974044 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c86519f-145a-43ca-8549-eb28dcf970e4 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:17.975691 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a8967f29-d61e-43d9-9d58-fec6838f7afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:17.984590 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 142/286] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:17:17 2022] GET /identity/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:17.991809 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86519f-145a-43ca-8549-eb28dcf970e4 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:18.087520 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 145/287] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:17:18.145598 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:18.171499 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86519f-145a-43ca-8549-eb28dcf970e4 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:18.182961 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8967f29-d61e-43d9-9d58-fec6838f7afc None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:18.221499 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07380fc-fa73-467a-ab4a-bfed8a44d12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:18.223943 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07380fc-fa73-467a-ab4a-bfed8a44d12b None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:18.224451 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07380fc-fa73-467a-ab4a-bfed8a44d12b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:18.226275 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07380fc-fa73-467a-ab4a-bfed8a44d12b None admin] PATH_INFO: `/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f/users/a0cb3047d64e4c64926ae71f71d6c667/roles/34a7a66da3654831a115d77e40a989da` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:18.229587 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07380fc-fa73-467a-ab4a-bfed8a44d12b None admin] RBAC: Authorizing `identity:create_grant(project_id=1264ceb6aa114fbf81b3d48cd8874c8f, user_id=a0cb3047d64e4c64926ae71f71d6c667, role_id=34a7a66da3654831a115d77e40a989da)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:18.232470 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07380fc-fa73-467a-ab4a-bfed8a44d12b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:18.264676 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 143/288] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 21:17:17 2022] PUT /identity/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f/users/a0cb3047d64e4c64926ae71f71d6c667/roles/34a7a66da3654831a115d77e40a989da => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:17:18.333461 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8967f29-d61e-43d9-9d58-fec6838f7afc None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:18.370647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:18.378947 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad62c9-e4c1-401f-b6d4-b45eea251df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:18.379846 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad62c9-e4c1-401f-b6d4-b45eea251df2 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:18.380131 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad62c9-e4c1-401f-b6d4-b45eea251df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:18.380409 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad62c9-e4c1-401f-b6d4-b45eea251df2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:18.397284 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad62c9-e4c1-401f-b6d4-b45eea251df2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:18.406499 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-b0ad62c9-e4c1-401f-b6d4-b45eea251df2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 21:17:18.406499 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:18.406499 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:18.406499 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:18.406499 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:18.406499 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:18.406499 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:18.406499 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:18.406499 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:18.406499 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:18.406499 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:18.406499 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:18.406499 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:18.406499 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:18.406499 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:18.406499 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:18.407491 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:18.407491 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:18.407491 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:18.407491 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:18.407491 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:18.407491 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:18.407491 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:18.407491 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:18.407491 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:18.407491 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:18.407491 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:18.407491 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:17:18.407491 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:18.407491 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:18.407491 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:18.407491 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:18.408271 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:18.408271 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:18.408271 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:18.408271 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:18.408271 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:18.408271 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:18.408271 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:18.408271 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:18.408271 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:18.408271 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:18.408271 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 21:17:18.408271 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:17:18.408271 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 146/289] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:17:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:17:18.415863 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07380fc-fa73-467a-ab4a-bfed8a44d12b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:18.580864 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07380fc-fa73-467a-ab4a-bfed8a44d12b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:18.630528 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9562cb9-5eb3-4ba9-973c-85e15cc1b6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:18.631209 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9562cb9-5eb3-4ba9-973c-85e15cc1b6bd None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:18.631488 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9562cb9-5eb3-4ba9-973c-85e15cc1b6bd None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:18.631757 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9562cb9-5eb3-4ba9-973c-85e15cc1b6bd None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:18.632512 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9562cb9-5eb3-4ba9-973c-85e15cc1b6bd None admin] RBAC: Adding query filter params (name=member) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:18.632813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9562cb9-5eb3-4ba9-973c-85e15cc1b6bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:18.634542 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9562cb9-5eb3-4ba9-973c-85e15cc1b6bd None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:18.643941 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 144/290] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 21:17:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:18.650404 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad62c9-e4c1-401f-b6d4-b45eea251df2 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:18.769326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad62c9-e4c1-401f-b6d4-b45eea251df2 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:18.809942 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488793b2-da57-4572-9b8a-6b7887b90e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:18.811241 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488793b2-da57-4572-9b8a-6b7887b90e49 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:18.811722 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488793b2-da57-4572-9b8a-6b7887b90e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:18.812238 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488793b2-da57-4572-9b8a-6b7887b90e49 None admin] PATH_INFO: `/v3/users/ddf67c24de3243b2b6cdb4db14ed1ad5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:18.813699 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488793b2-da57-4572-9b8a-6b7887b90e49 None admin] RBAC: Authorizing `identity:get_user(user_id=ddf67c24de3243b2b6cdb4db14ed1ad5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:18.815397 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488793b2-da57-4572-9b8a-6b7887b90e49 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:18.817612 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 147/291] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:17:18 2022] GET /identity/v3/users/ddf67c24de3243b2b6cdb4db14ed1ad5 => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:18.823362 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9562cb9-5eb3-4ba9-973c-85e15cc1b6bd None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:18.826598 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:18.972865 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9562cb9-5eb3-4ba9-973c-85e15cc1b6bd None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:19.028913 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365ff70-9729-4a3a-aa90-fc981487a8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:19.029584 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365ff70-9729-4a3a-aa90-fc981487a8c7 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:19.029864 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365ff70-9729-4a3a-aa90-fc981487a8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:19.030136 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365ff70-9729-4a3a-aa90-fc981487a8c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:19.119222 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2365ff70-9729-4a3a-aa90-fc981487a8c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:19.120754 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2365ff70-9729-4a3a-aa90-fc981487a8c7 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:19.126979 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:19.240963 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 145/292] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:17:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 21:17:19.247465 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488793b2-da57-4572-9b8a-6b7887b90e49 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:19.327971 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:19.384829 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488793b2-da57-4572-9b8a-6b7887b90e49 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:19.431573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85a0e3-e256-4b35-a433-e4e3b352c5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:19.432455 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85a0e3-e256-4b35-a433-e4e3b352c5d0 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:19.432736 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85a0e3-e256-4b35-a433-e4e3b352c5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:19.433013 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85a0e3-e256-4b35-a433-e4e3b352c5d0 None admin] PATH_INFO: `/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:19.434258 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85a0e3-e256-4b35-a433-e4e3b352c5d0 None admin] RBAC: Authorizing `identity:get_project(project_id=1264ceb6aa114fbf81b3d48cd8874c8f)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:19.435746 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85a0e3-e256-4b35-a433-e4e3b352c5d0 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:19.437761 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 148/293] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:17:19 2022] GET /identity/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:19.443715 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365ff70-9729-4a3a-aa90-fc981487a8c7 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:19.599785 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365ff70-9729-4a3a-aa90-fc981487a8c7 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:19.601573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:19.661691 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d823a-e096-49d6-bc50-a60bbd92c06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:19.662433 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d823a-e096-49d6-bc50-a60bbd92c06b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:19.662718 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d823a-e096-49d6-bc50-a60bbd92c06b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:19.662992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d823a-e096-49d6-bc50-a60bbd92c06b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:19.663880 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d823a-e096-49d6-bc50-a60bbd92c06b None admin] RBAC: Adding query filter params (role.id=70699fb0eae7441999835fa3b50350b3, scope.project.id=1264ceb6aa114fbf81b3d48cd8874c8f, user.id=ddf67c24de3243b2b6cdb4db14ed1ad5) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:19.664194 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d823a-e096-49d6-bc50-a60bbd92c06b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:19.665915 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d823a-e096-49d6-bc50-a60bbd92c06b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:19.671923 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ff78b-5cce-47a4-a930-ce7df15c2def None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:19.672219 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ff78b-5cce-47a4-a930-ce7df15c2def None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:19.672473 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ff78b-5cce-47a4-a930-ce7df15c2def None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:19.674184 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 149/294] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:19.677084 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 146/295] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:17:19 2022] GET /identity/v3/role_assignments?user.id=ddf67c24de3243b2b6cdb4db14ed1ad5&scope.project.id=1264ceb6aa114fbf81b3d48cd8874c8f&role.id=70699fb0eae7441999835fa3b50350b3 => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:19.679493 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559eb57-0145-481a-8f37-a4750e4161a6 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:19.679781 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559eb57-0145-481a-8f37-a4750e4161a6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:19.680059 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559eb57-0145-481a-8f37-a4750e4161a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:19.715498 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-7559eb57-0145-481a-8f37-a4750e4161a6 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:19.716714 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7559eb57-0145-481a-8f37-a4750e4161a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:19.816910 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 150/296] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:17:19.851198 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d823a-e096-49d6-bc50-a60bbd92c06b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:19.943546 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9d4fa-65a2-4324-8861-9882d1f1a7ee None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:19.943857 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9d4fa-65a2-4324-8861-9882d1f1a7ee None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:19.944098 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9d4fa-65a2-4324-8861-9882d1f1a7ee None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:19.946737 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 151/297] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:17:19.955678 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08b237-d53c-45c2-8924-1a406c1a231d None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:19.955963 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08b237-d53c-45c2-8924-1a406c1a231d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:19.956241 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08b237-d53c-45c2-8924-1a406c1a231d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:19.980559 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d823a-e096-49d6-bc50-a60bbd92c06b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:19.983876 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-1d08b237-d53c-45c2-8924-1a406c1a231d None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:19.985049 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1d08b237-d53c-45c2-8924-1a406c1a231d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:20.020657 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd414011-ff17-487d-a773-ad23ee8ffb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:20.021466 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd414011-ff17-487d-a773-ad23ee8ffb4c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:20.021748 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd414011-ff17-487d-a773-ad23ee8ffb4c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:20.022073 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd414011-ff17-487d-a773-ad23ee8ffb4c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:20.023254 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd414011-ff17-487d-a773-ad23ee8ffb4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:20.024981 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd414011-ff17-487d-a773-ad23ee8ffb4c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:20.026985 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 147/298] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:17:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:20.033048 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd414011-ff17-487d-a773-ad23ee8ffb4c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:20.042580 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:20.068007 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 152/299] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:17:20.084534 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:20.095768 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d08b237-d53c-45c2-8924-1a406c1a231d None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:20.187614 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd414011-ff17-487d-a773-ad23ee8ffb4c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:20.237015 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ec1fa3-fb5a-42bb-954c-d585b0f0d331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:20.237704 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ec1fa3-fb5a-42bb-954c-d585b0f0d331 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:20.238229 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ec1fa3-fb5a-42bb-954c-d585b0f0d331 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:20.238539 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ec1fa3-fb5a-42bb-954c-d585b0f0d331 None admin] PATH_INFO: `/v3/groups` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:20.239775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ec1fa3-fb5a-42bb-954c-d585b0f0d331 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:20.241515 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ec1fa3-fb5a-42bb-954c-d585b0f0d331 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:20.243389 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d08b237-d53c-45c2-8924-1a406c1a231d None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:20.264070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 148/300] 10.222.0.65 () {60 vars in 1134 bytes} [Thu May 19 21:17:20 2022] POST /identity/v3/groups => generated 229 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:17:20.288669 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2823c0a-85d0-4cc0-9505-8e3f3298dc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:20.289555 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2823c0a-85d0-4cc0-9505-8e3f3298dc11 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:20.289904 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2823c0a-85d0-4cc0-9505-8e3f3298dc11 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:20.290334 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2823c0a-85d0-4cc0-9505-8e3f3298dc11 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:20.303449 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2823c0a-85d0-4cc0-9505-8e3f3298dc11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:20.311839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-c2823c0a-85d0-4cc0-9505-8e3f3298dc11 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 21:17:20.311839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:20.311839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:20.311839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:20.311839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:20.311839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:20.311839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:20.311839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:20.311839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:20.311839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:20.311839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:20.311839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:20.311839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:20.311839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:20.311839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:20.311839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:20.312836 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:20.312836 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:20.312836 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:20.312836 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:20.312836 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:20.312836 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:20.312836 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:20.312836 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:20.312836 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:20.312836 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:20.312836 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:20.312836 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:17:20.312836 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:20.312836 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:20.312836 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:20.312836 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:20.314957 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:20.314957 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:20.314957 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:20.314957 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:20.314957 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:20.314957 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:20.314957 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:20.314957 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:20.314957 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:20.314957 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:20.314957 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 21:17:20.314957 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:17:20.314957 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 153/301] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:17:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:20.322114 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ec1fa3-fb5a-42bb-954c-d585b0f0d331 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:20.343904 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e1becb-e5b9-4497-9aac-a511ba82aeb3 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:20.344527 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e1becb-e5b9-4497-9aac-a511ba82aeb3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:20.345067 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e1becb-e5b9-4497-9aac-a511ba82aeb3 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:20.347046 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 154/302] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:20.354787 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166da142-4d83-40e7-a948-e213dab32c83 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:20.355094 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166da142-4d83-40e7-a948-e213dab32c83 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:20.355346 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166da142-4d83-40e7-a948-e213dab32c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:20.374112 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-166da142-4d83-40e7-a948-e213dab32c83 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:20.375376 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-166da142-4d83-40e7-a948-e213dab32c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:20.437929 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:20.446162 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:20.461268 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ec1fa3-fb5a-42bb-954c-d585b0f0d331 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:20.476033 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 155/303] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:17:20.500381 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166da142-4d83-40e7-a948-e213dab32c83 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:20.506349 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1474a1-c0af-4ad9-8076-340b33d8b5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:20.507047 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1474a1-c0af-4ad9-8076-340b33d8b5cd None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:20.507347 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1474a1-c0af-4ad9-8076-340b33d8b5cd None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:20.507619 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1474a1-c0af-4ad9-8076-340b33d8b5cd None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:20.508481 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1474a1-c0af-4ad9-8076-340b33d8b5cd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:20.508882 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1474a1-c0af-4ad9-8076-340b33d8b5cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:20.511076 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1474a1-c0af-4ad9-8076-340b33d8b5cd None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:20.523494 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 149/304] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 21:17:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:20.529762 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1474a1-c0af-4ad9-8076-340b33d8b5cd None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:20.631703 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166da142-4d83-40e7-a948-e213dab32c83 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:20.662841 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1474a1-c0af-4ad9-8076-340b33d8b5cd None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:20.670832 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7978f4b-526a-4f8b-a0c5-b42eba66a528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:20.671855 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7978f4b-526a-4f8b-a0c5-b42eba66a528 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:20.672200 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7978f4b-526a-4f8b-a0c5-b42eba66a528 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:20.672539 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7978f4b-526a-4f8b-a0c5-b42eba66a528 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:20.684445 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7978f4b-526a-4f8b-a0c5-b42eba66a528 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:20.693292 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-a7978f4b-526a-4f8b-a0c5-b42eba66a528 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 21:17:20.693292 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:20.693292 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:20.693292 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:20.693292 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:20.693292 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:20.693292 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:20.693292 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:20.693292 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:20.693292 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:20.693292 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:20.693292 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:20.693292 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:20.693292 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:20.693292 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:20.693292 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:20.694151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:20.694151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:20.694151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:20.694151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:20.694151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:20.694151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:20.694151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:20.694151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:20.694151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:20.694151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:20.694151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:20.694151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:17:20.694151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:20.694151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:20.694151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:20.694151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:20.694988 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:20.694988 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:20.694988 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:20.694988 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:20.694988 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:20.694988 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:20.694988 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:20.694988 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:20.694988 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:20.694988 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:20.694988 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 21:17:20.694988 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:17:20.694988 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 156/305] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:17:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:20.701427 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7978f4b-526a-4f8b-a0c5-b42eba66a528 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:20.707243 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc9b09-30c1-40c2-a04d-f679c1b61112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:20.708142 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc9b09-30c1-40c2-a04d-f679c1b61112 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:20.708419 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc9b09-30c1-40c2-a04d-f679c1b61112 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:20.708694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc9b09-30c1-40c2-a04d-f679c1b61112 None admin] PATH_INFO: `/v3/users/199b117f033f464fb07b2e658eb28c46` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:20.730034 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dc9b09-30c1-40c2-a04d-f679c1b61112 None admin] RBAC: Authorizing `identity:get_user(user_id=199b117f033f464fb07b2e658eb28c46)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:20.730393 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dc9b09-30c1-40c2-a04d-f679c1b61112 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:20.732684 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 150/306] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:17:20 2022] GET /identity/v3/users/199b117f033f464fb07b2e658eb28c46 => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:17:20.738303 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc9b09-30c1-40c2-a04d-f679c1b61112 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:20.746838 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:20.758731 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:20.819442 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7978f4b-526a-4f8b-a0c5-b42eba66a528 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:20.846489 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc9b09-30c1-40c2-a04d-f679c1b61112 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:20.859433 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadde770-65d1-465e-b51d-33b0ad9fbd1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:20.860452 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadde770-65d1-465e-b51d-33b0ad9fbd1a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:20.860931 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadde770-65d1-465e-b51d-33b0ad9fbd1a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:20.861439 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadde770-65d1-465e-b51d-33b0ad9fbd1a None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:20.862452 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadde770-65d1-465e-b51d-33b0ad9fbd1a None admin] RBAC: Adding query filter params (name=member) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:20.862959 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadde770-65d1-465e-b51d-33b0ad9fbd1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:20.864562 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadde770-65d1-465e-b51d-33b0ad9fbd1a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:20.879446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 157/307] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 21:17:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:20.880686 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadde770-65d1-465e-b51d-33b0ad9fbd1a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:20.888127 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e582f68a-9327-4d7b-bb8c-9b24f581d87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:20.888805 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e582f68a-9327-4d7b-bb8c-9b24f581d87a None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:20.889372 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e582f68a-9327-4d7b-bb8c-9b24f581d87a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:20.889644 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e582f68a-9327-4d7b-bb8c-9b24f581d87a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:20.890519 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e582f68a-9327-4d7b-bb8c-9b24f581d87a None admin] RBAC: Adding query filter params (role.id=34a7a66da3654831a115d77e40a989da, user.id=199b117f033f464fb07b2e658eb28c46, scope.system=all) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:20.890829 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e582f68a-9327-4d7b-bb8c-9b24f581d87a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:20.892280 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e582f68a-9327-4d7b-bb8c-9b24f581d87a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:20.905689 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 151/308] 10.222.0.65 () {56 vars in 1308 bytes} [Thu May 19 21:17:20 2022] GET /identity/v3/role_assignments?user.id=199b117f033f464fb07b2e658eb28c46&scope.system=all&role.id=34a7a66da3654831a115d77e40a989da => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:17:20.998217 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:21.003947 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadde770-65d1-465e-b51d-33b0ad9fbd1a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:21.041844 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d2f84-9ca6-450f-9b8e-9ef6af38f3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:21.043190 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d2f84-9ca6-450f-9b8e-9ef6af38f3c4 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:21.043722 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d2f84-9ca6-450f-9b8e-9ef6af38f3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:21.044234 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d2f84-9ca6-450f-9b8e-9ef6af38f3c4 None admin] PATH_INFO: `/v3/users/cbd78a2644b14db8a3056e7d57ebf032` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:21.045832 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460d2f84-9ca6-450f-9b8e-9ef6af38f3c4 None admin] RBAC: Authorizing `identity:get_user(user_id=cbd78a2644b14db8a3056e7d57ebf032)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:21.047611 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460d2f84-9ca6-450f-9b8e-9ef6af38f3c4 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:21.049643 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 158/309] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:17:20 2022] GET /identity/v3/users/cbd78a2644b14db8a3056e7d57ebf032 => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:21.055873 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e582f68a-9327-4d7b-bb8c-9b24f581d87a None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:21.146270 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:21.169705 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e582f68a-9327-4d7b-bb8c-9b24f581d87a None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:21.207854 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aaa14e-348b-40c9-972b-fcec877fc58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:21.209140 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aaa14e-348b-40c9-972b-fcec877fc58c None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:21.209620 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aaa14e-348b-40c9-972b-fcec877fc58c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:21.210091 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aaa14e-348b-40c9-972b-fcec877fc58c None admin] PATH_INFO: `/v3/system/users/cbd78a2644b14db8a3056e7d57ebf032/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:21.211889 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aaa14e-348b-40c9-972b-fcec877fc58c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=cbd78a2644b14db8a3056e7d57ebf032, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:21.213643 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aaa14e-348b-40c9-972b-fcec877fc58c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:21.228141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 152/310] 10.222.0.65 () {58 vars in 1334 bytes} [Thu May 19 21:17:21 2022] PUT /identity/v3/system/users/cbd78a2644b14db8a3056e7d57ebf032/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:17:21.325932 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa2645-c7f1-4cfa-89df-60f75530d676 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:21.326468 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa2645-c7f1-4cfa-89df-60f75530d676 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:21.326468 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa2645-c7f1-4cfa-89df-60f75530d676 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:21.328009 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 159/311] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:21.334102 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5d90d-d753-4646-b408-20205aa03e69 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:21.334673 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5d90d-d753-4646-b408-20205aa03e69 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:21.335150 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e5d90d-d753-4646-b408-20205aa03e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:21.355069 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-76e5d90d-d753-4646-b408-20205aa03e69 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:21.356650 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-76e5d90d-d753-4646-b408-20205aa03e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:21.440732 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 153/312] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:17:21.463854 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa2645-c7f1-4cfa-89df-60f75530d676 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:21.574933 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa2645-c7f1-4cfa-89df-60f75530d676 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:21.611898 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e680a69-a7eb-44cc-b749-6a172638fcae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:21.612716 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e680a69-a7eb-44cc-b749-6a172638fcae None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:21.612994 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e680a69-a7eb-44cc-b749-6a172638fcae None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:21.613262 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e680a69-a7eb-44cc-b749-6a172638fcae None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:21.631430 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e680a69-a7eb-44cc-b749-6a172638fcae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:21.641338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-2e680a69-a7eb-44cc-b749-6a172638fcae None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 21:17:21.641338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:21.641338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:21.641338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:21.641338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:21.641338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:21.641338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:21.641338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:21.641338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:21.641338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:21.641338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:21.641338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:21.641338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:21.641338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:21.641338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:21.641338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:21.642305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:21.642305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:21.642305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:21.642305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:21.642305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:21.642305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:21.642305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:21.642305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:21.642305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:21.642305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:21.642305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:21.642305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:17:21.642305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:21.642305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:21.642305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:21.642305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:21.643113 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:21.643113 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:21.643113 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:21.643113 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:21.643113 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:21.643113 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:21.643113 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:21.643113 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:21.643113 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:21.643113 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:21.643113 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 21:17:21.643113 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:17:21.643113 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 160/313] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:17:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:21.651069 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5d90d-d753-4646-b408-20205aa03e69 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:21.731059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:21.768505 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e5d90d-d753-4646-b408-20205aa03e69 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:21.808246 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3b321b-aaf4-4141-ac5c-733f28dc286d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:21.808942 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b321b-aaf4-4141-ac5c-733f28dc286d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:21.809221 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b321b-aaf4-4141-ac5c-733f28dc286d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:21.809489 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b321b-aaf4-4141-ac5c-733f28dc286d None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:21.810271 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3b321b-aaf4-4141-ac5c-733f28dc286d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:21.810575 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3b321b-aaf4-4141-ac5c-733f28dc286d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:21.812018 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3b321b-aaf4-4141-ac5c-733f28dc286d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:21.821922 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 154/314] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 21:17:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:21.831412 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e680a69-a7eb-44cc-b749-6a172638fcae None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:21.876088 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:21.958087 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e680a69-a7eb-44cc-b749-6a172638fcae None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:21.996756 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec743bb-4563-43e1-a96a-0af6d5a02a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:21.998005 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec743bb-4563-43e1-a96a-0af6d5a02a45 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:21.998528 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec743bb-4563-43e1-a96a-0af6d5a02a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:21.999086 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec743bb-4563-43e1-a96a-0af6d5a02a45 None admin] PATH_INFO: `/v3/users/a0cb3047d64e4c64926ae71f71d6c667` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:22.000619 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec743bb-4563-43e1-a96a-0af6d5a02a45 None admin] RBAC: Authorizing `identity:get_user(user_id=a0cb3047d64e4c64926ae71f71d6c667)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:22.002352 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec743bb-4563-43e1-a96a-0af6d5a02a45 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:22.004642 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 161/315] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:17:21 2022] GET /identity/v3/users/a0cb3047d64e4c64926ae71f71d6c667 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:22.010443 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3b321b-aaf4-4141-ac5c-733f28dc286d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:22.128824 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3b321b-aaf4-4141-ac5c-733f28dc286d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:22.164692 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611e9979-0fbf-4d7c-af28-81c926209fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:22.165514 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611e9979-0fbf-4d7c-af28-81c926209fec None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:22.165794 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611e9979-0fbf-4d7c-af28-81c926209fec None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:22.166064 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611e9979-0fbf-4d7c-af28-81c926209fec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:22.168931 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:22.238039 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611e9979-0fbf-4d7c-af28-81c926209fec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:22.240837 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611e9979-0fbf-4d7c-af28-81c926209fec None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:22.345502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 155/316] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:17:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:17:22.351349 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec743bb-4563-43e1-a96a-0af6d5a02a45 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:22.470775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:22.478387 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec743bb-4563-43e1-a96a-0af6d5a02a45 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:22.518604 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f4cc0-28c7-4bd0-9105-e0fb4dc30bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:22.519484 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f4cc0-28c7-4bd0-9105-e0fb4dc30bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:22.519768 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f4cc0-28c7-4bd0-9105-e0fb4dc30bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:22.520047 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f4cc0-28c7-4bd0-9105-e0fb4dc30bd9 None admin] PATH_INFO: `/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:22.521260 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216f4cc0-28c7-4bd0-9105-e0fb4dc30bd9 None admin] RBAC: Authorizing `identity:get_project(project_id=1264ceb6aa114fbf81b3d48cd8874c8f)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:22.522741 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216f4cc0-28c7-4bd0-9105-e0fb4dc30bd9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:22.524785 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 162/317] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:17:22 2022] GET /identity/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:22.530760 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611e9979-0fbf-4d7c-af28-81c926209fec None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:22.570468 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:22.641420 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611e9979-0fbf-4d7c-af28-81c926209fec None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:22.679631 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67784d2-db73-43d6-bc57-fe7a231e8412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:22.680694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67784d2-db73-43d6-bc57-fe7a231e8412 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:22.680983 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67784d2-db73-43d6-bc57-fe7a231e8412 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:22.681260 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67784d2-db73-43d6-bc57-fe7a231e8412 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:22.682102 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67784d2-db73-43d6-bc57-fe7a231e8412 None admin] RBAC: Adding query filter params (role.id=34a7a66da3654831a115d77e40a989da, scope.project.id=1264ceb6aa114fbf81b3d48cd8874c8f, user.id=a0cb3047d64e4c64926ae71f71d6c667) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:22.682442 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67784d2-db73-43d6-bc57-fe7a231e8412 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:22.683912 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67784d2-db73-43d6-bc57-fe7a231e8412 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:22.694364 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 156/318] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:17:22 2022] GET /identity/v3/role_assignments?user.id=a0cb3047d64e4c64926ae71f71d6c667&scope.project.id=1264ceb6aa114fbf81b3d48cd8874c8f&role.id=34a7a66da3654831a115d77e40a989da => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:22.776198 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fcc904-f216-43ea-91b7-0998d376cfb1 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:22.776449 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fcc904-f216-43ea-91b7-0998d376cfb1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:22.776773 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fcc904-f216-43ea-91b7-0998d376cfb1 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:22.778450 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 163/319] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:22.784249 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fdd7cd-a457-4fd2-b62e-569683b496c9 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:22.784648 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fdd7cd-a457-4fd2-b62e-569683b496c9 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:22.784883 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fdd7cd-a457-4fd2-b62e-569683b496c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:22.803654 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-e4fdd7cd-a457-4fd2-b62e-569683b496c9 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:22.804894 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e4fdd7cd-a457-4fd2-b62e-569683b496c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:22.891274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 157/320] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:17:22.915605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fcc904-f216-43ea-91b7-0998d376cfb1 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:23.034583 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fcc904-f216-43ea-91b7-0998d376cfb1 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:23.073794 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34998deb-2db6-47b4-9e7a-53486207fab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:23.074638 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34998deb-2db6-47b4-9e7a-53486207fab3 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:23.074919 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34998deb-2db6-47b4-9e7a-53486207fab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:23.075182 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34998deb-2db6-47b4-9e7a-53486207fab3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:23.076422 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34998deb-2db6-47b4-9e7a-53486207fab3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:23.077921 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34998deb-2db6-47b4-9e7a-53486207fab3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:23.079647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 164/321] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:17:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:23.085358 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fdd7cd-a457-4fd2-b62e-569683b496c9 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:23.113876 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:23.194785 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fdd7cd-a457-4fd2-b62e-569683b496c9 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:23.230570 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488e5b6-77c1-4c8d-8142-01ea9d068cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:23.231242 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488e5b6-77c1-4c8d-8142-01ea9d068cde None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:23.231523 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488e5b6-77c1-4c8d-8142-01ea9d068cde None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:23.231792 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488e5b6-77c1-4c8d-8142-01ea9d068cde None admin] PATH_INFO: `/v3/groups` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:23.232472 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3488e5b6-77c1-4c8d-8142-01ea9d068cde None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:23.233932 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3488e5b6-77c1-4c8d-8142-01ea9d068cde None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:23.249821 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 158/322] 10.222.0.65 () {60 vars in 1134 bytes} [Thu May 19 21:17:23 2022] POST /identity/v3/groups => generated 226 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:17:23.316801 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f3eee-4463-4953-b724-c13bac35c2e7 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:23.317001 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f3eee-4463-4953-b724-c13bac35c2e7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:23.318072 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f3eee-4463-4953-b724-c13bac35c2e7 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:23.319868 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 165/323] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:23.326010 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc2c4b4-aa13-43e9-92c1-7338d813fd9c None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:23.326364 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc2c4b4-aa13-43e9-92c1-7338d813fd9c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:23.326601 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc2c4b4-aa13-43e9-92c1-7338d813fd9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:23.345293 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-0dc2c4b4-aa13-43e9-92c1-7338d813fd9c None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:23.346570 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0dc2c4b4-aa13-43e9-92c1-7338d813fd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:23.428389 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 159/324] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:17:23.450218 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1f3eee-4463-4953-b724-c13bac35c2e7 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:23.498001 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:23.570667 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c0a1d-6d9b-4048-b900-392122505efc None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:23.570973 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c0a1d-6d9b-4048-b900-392122505efc None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:23.571235 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c0a1d-6d9b-4048-b900-392122505efc None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:23.572906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 160/325] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:17:23.578607 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca3042-9593-43fd-89e1-548e882c0609 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:23.578607 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca3042-9593-43fd-89e1-548e882c0609 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:23.578905 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca3042-9593-43fd-89e1-548e882c0609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:23.584993 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1f3eee-4463-4953-b724-c13bac35c2e7 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:23.602852 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-d1ca3042-9593-43fd-89e1-548e882c0609 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:23.604311 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d1ca3042-9593-43fd-89e1-548e882c0609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:23.620634 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:23.624517 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf36826-12f1-4f7e-9143-63e1b672ddb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:23.625466 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf36826-12f1-4f7e-9143-63e1b672ddb2 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:23.625587 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf36826-12f1-4f7e-9143-63e1b672ddb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:23.625863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf36826-12f1-4f7e-9143-63e1b672ddb2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:23.642604 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf36826-12f1-4f7e-9143-63e1b672ddb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:23.651157 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-2cf36826-12f1-4f7e-9143-63e1b672ddb2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 21:17:23.651157 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:23.651157 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:23.651157 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:23.651157 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:23.651157 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:23.651157 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:23.651157 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:23.651157 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:23.651157 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:23.651157 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:23.651157 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:23.651157 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:23.651157 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:23.651157 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:23.651157 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:23.652144 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:23.652144 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:23.652144 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:23.652144 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:23.652144 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:23.652144 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:23.652144 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:23.652144 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:23.652144 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:23.652144 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:23.652144 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:23.652144 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:17:23.652144 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:23.652144 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:23.652144 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:23.652144 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:23.652971 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:23.652971 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:23.652971 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:23.652971 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:23.652971 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:23.652971 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:23.652971 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:23.652971 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:23.652971 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:23.652971 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:23.652971 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 21:17:23.652971 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:17:23.652971 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 166/326] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:17:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:23.660700 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf36826-12f1-4f7e-9143-63e1b672ddb2 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:23.709325 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 161/327] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:17:23.750718 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca3042-9593-43fd-89e1-548e882c0609 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:23.783908 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:23.807358 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf36826-12f1-4f7e-9143-63e1b672ddb2 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:23.878461 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e1a017-9e2c-4767-a338-dd9a6f5ad38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:23.879548 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1a017-9e2c-4767-a338-dd9a6f5ad38c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:23.880230 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1a017-9e2c-4767-a338-dd9a6f5ad38c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:23.881215 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1a017-9e2c-4767-a338-dd9a6f5ad38c None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:23.883883 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e1a017-9e2c-4767-a338-dd9a6f5ad38c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:23.884480 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e1a017-9e2c-4767-a338-dd9a6f5ad38c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:23.886148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e1a017-9e2c-4767-a338-dd9a6f5ad38c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:23.901933 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca3042-9593-43fd-89e1-548e882c0609 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:23.904548 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 167/328] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 21:17:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:23.913296 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e1a017-9e2c-4767-a338-dd9a6f5ad38c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:23.997727 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1ef7c-e369-4ff5-98e2-8647a819f029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:23.998599 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1ef7c-e369-4ff5-98e2-8647a819f029 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:23.998881 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1ef7c-e369-4ff5-98e2-8647a819f029 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:23.999155 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1ef7c-e369-4ff5-98e2-8647a819f029 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:24.018602 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d1ef7c-e369-4ff5-98e2-8647a819f029 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:24.037170 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-f3d1ef7c-e369-4ff5-98e2-8647a819f029 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 21:17:24.037170 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:24.037170 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:24.037170 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:24.037170 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:24.037170 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:24.037170 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:24.037170 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:24.037170 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:24.037170 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:24.037170 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:24.037170 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:24.037170 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:24.037170 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:24.037170 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:24.037170 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:24.038945 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:24.038945 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:24.038945 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:24.038945 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:24.038945 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:24.038945 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:24.038945 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:24.038945 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:24.038945 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:24.038945 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:24.038945 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:24.038945 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:24.038945 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:24.038945 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:24.038945 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:24.038945 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:24.040528 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:24.040528 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:24.040528 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:24.040528 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:24.040528 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:24.040528 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:24.040528 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:24.040528 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:24.040528 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:24.040528 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:24.040528 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 21:17:24.040528 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:24.040528 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 162/329] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:17:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:24.045180 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1ef7c-e369-4ff5-98e2-8647a819f029 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:24.062427 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:24.120582 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e1a017-9e2c-4767-a338-dd9a6f5ad38c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:24.157344 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f977caff-70df-4f8a-b64e-bc3eb32e15fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:24.158355 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977caff-70df-4f8a-b64e-bc3eb32e15fa None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:24.161573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977caff-70df-4f8a-b64e-bc3eb32e15fa None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:24.161573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977caff-70df-4f8a-b64e-bc3eb32e15fa None admin] PATH_INFO: `/v3/users/199b117f033f464fb07b2e658eb28c46` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:24.161573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f977caff-70df-4f8a-b64e-bc3eb32e15fa None admin] RBAC: Authorizing `identity:get_user(user_id=199b117f033f464fb07b2e658eb28c46)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:24.161573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f977caff-70df-4f8a-b64e-bc3eb32e15fa None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:24.163618 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 168/330] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:17:23 2022] GET /identity/v3/users/199b117f033f464fb07b2e658eb28c46 => generated 293 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:24.169283 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f977caff-70df-4f8a-b64e-bc3eb32e15fa None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:24.193708 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:24.279734 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f977caff-70df-4f8a-b64e-bc3eb32e15fa None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:24.300282 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1ef7c-e369-4ff5-98e2-8647a819f029 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:24.315352 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918531f5-988c-4fc3-aa34-6e3a910a020a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:24.316238 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918531f5-988c-4fc3-aa34-6e3a910a020a None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:24.316521 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918531f5-988c-4fc3-aa34-6e3a910a020a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:24.317240 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918531f5-988c-4fc3-aa34-6e3a910a020a None admin] PATH_INFO: `/v3/system/users/199b117f033f464fb07b2e658eb28c46/roles/34a7a66da3654831a115d77e40a989da` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:24.319377 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918531f5-988c-4fc3-aa34-6e3a910a020a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=199b117f033f464fb07b2e658eb28c46, role_id=34a7a66da3654831a115d77e40a989da)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:24.320792 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918531f5-988c-4fc3-aa34-6e3a910a020a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:24.337666 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 169/331] 10.222.0.65 () {58 vars in 1334 bytes} [Thu May 19 21:17:24 2022] PUT /identity/v3/system/users/199b117f033f464fb07b2e658eb28c46/roles/34a7a66da3654831a115d77e40a989da => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:17:24.368674 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d259a-942a-4160-9a37-085c6170e971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:24.369427 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d259a-942a-4160-9a37-085c6170e971 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:24.369705 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d259a-942a-4160-9a37-085c6170e971 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:24.369978 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d259a-942a-4160-9a37-085c6170e971 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:24.370872 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d259a-942a-4160-9a37-085c6170e971 None admin] RBAC: Adding query filter params (name=member) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:24.371142 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d259a-942a-4160-9a37-085c6170e971 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:24.372537 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d259a-942a-4160-9a37-085c6170e971 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:24.387802 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 163/332] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 21:17:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:24.393260 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918531f5-988c-4fc3-aa34-6e3a910a020a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:24.509205 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:24.529957 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918531f5-988c-4fc3-aa34-6e3a910a020a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:24.575404 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549aec1e-0a06-40f8-aeb6-485c7da32889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:24.576395 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549aec1e-0a06-40f8-aeb6-485c7da32889 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:24.576676 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549aec1e-0a06-40f8-aeb6-485c7da32889 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:24.576945 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549aec1e-0a06-40f8-aeb6-485c7da32889 None admin] PATH_INFO: `/v3/users/cbd78a2644b14db8a3056e7d57ebf032` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:24.578484 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549aec1e-0a06-40f8-aeb6-485c7da32889 None admin] RBAC: Authorizing `identity:get_user(user_id=cbd78a2644b14db8a3056e7d57ebf032)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:24.580085 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549aec1e-0a06-40f8-aeb6-485c7da32889 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:24.581916 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 170/333] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:17:24 2022] GET /identity/v3/users/cbd78a2644b14db8a3056e7d57ebf032 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:24.588735 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d259a-942a-4160-9a37-085c6170e971 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:24.679392 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:24.719214 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d259a-942a-4160-9a37-085c6170e971 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:24.769317 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d932aa8-9b4c-4a3c-bb2d-b5481edb0ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:24.770616 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d932aa8-9b4c-4a3c-bb2d-b5481edb0ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:24.771164 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d932aa8-9b4c-4a3c-bb2d-b5481edb0ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:24.771647 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d932aa8-9b4c-4a3c-bb2d-b5481edb0ab5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:24.772782 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d932aa8-9b4c-4a3c-bb2d-b5481edb0ab5 None admin] RBAC: Adding query filter params (role.id=70699fb0eae7441999835fa3b50350b3, user.id=cbd78a2644b14db8a3056e7d57ebf032, scope.system=all) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:24.773358 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d932aa8-9b4c-4a3c-bb2d-b5481edb0ab5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:24.775344 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d932aa8-9b4c-4a3c-bb2d-b5481edb0ab5 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:24.794320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 164/334] 10.222.0.65 () {56 vars in 1308 bytes} [Thu May 19 21:17:24 2022] GET /identity/v3/role_assignments?user.id=cbd78a2644b14db8a3056e7d57ebf032&scope.system=all&role.id=70699fb0eae7441999835fa3b50350b3 => generated 510 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:26.924339 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745e006-96f2-4040-ae13-55de0867239f None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:26.925357 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745e006-96f2-4040-ae13-55de0867239f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:26.925831 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745e006-96f2-4040-ae13-55de0867239f None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:26.927507 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 171/335] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:26.933903 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8588e-8c26-4c9f-adc1-63a79a739195 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:26.935116 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8588e-8c26-4c9f-adc1-63a79a739195 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:26.935588 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8588e-8c26-4c9f-adc1-63a79a739195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:26.940360 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5db9c6-77ff-482d-8a73-929c35b4e259 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:26.940647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5db9c6-77ff-482d-8a73-929c35b4e259 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:26.940923 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5db9c6-77ff-482d-8a73-929c35b4e259 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:26.942774 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 172/336] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:26.959843 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-95a8588e-8c26-4c9f-adc1-63a79a739195 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:26.961508 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-95a8588e-8c26-4c9f-adc1-63a79a739195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:26.961695 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf846d2-e30c-4710-bc25-dbc453e89c89 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:26.961981 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf846d2-e30c-4710-bc25-dbc453e89c89 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:26.962287 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf846d2-e30c-4710-bc25-dbc453e89c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:26.995049 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-6cf846d2-e30c-4710-bc25-dbc453e89c89 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:26.996609 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6cf846d2-e30c-4710-bc25-dbc453e89c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:27.110110 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 173/337] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:17:27.132439 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 165/338] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:17:27.143418 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf846d2-e30c-4710-bc25-dbc453e89c89 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:27.181269 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a8588e-8c26-4c9f-adc1-63a79a739195 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:27.288420 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:27.316934 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf846d2-e30c-4710-bc25-dbc453e89c89 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:27.354673 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:27.383142 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d982fe-c309-4234-b96b-687a1e147121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:27.384029 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d982fe-c309-4234-b96b-687a1e147121 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:27.384309 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d982fe-c309-4234-b96b-687a1e147121 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:27.384673 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d982fe-c309-4234-b96b-687a1e147121 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:27.403432 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d982fe-c309-4234-b96b-687a1e147121 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:27.411733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-90d982fe-c309-4234-b96b-687a1e147121 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:17:27.411733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:27.411733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:27.411733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:27.411733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:27.411733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:27.411733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:27.411733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:27.411733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:27.411733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:27.411733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:27.411733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:27.411733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:27.411733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:27.411733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:27.411733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:27.412677 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:27.412677 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:27.412677 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:27.412677 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:27.412677 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:27.412677 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:27.412677 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:27.412677 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:27.412677 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:27.412677 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:27.412677 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:27.412677 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:17:27.412677 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:27.412677 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:27.412677 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:27.412677 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:27.413502 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:27.413502 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:27.413502 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:27.413502 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:27.413502 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:27.413502 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:27.413502 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:27.413502 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:27.413502 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:27.413502 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:27.413502 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:17:27.413502 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:17:27.413502 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 174/339] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 21:17:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:27.417852 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb558a-cb4f-42f4-a1f1-c43f9eeb85a1 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:27.418142 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb558a-cb4f-42f4-a1f1-c43f9eeb85a1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:27.418442 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb558a-cb4f-42f4-a1f1-c43f9eeb85a1 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:27.419827 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 175/340] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:27.422661 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb558a-cb4f-42f4-a1f1-c43f9eeb85a1 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:27.440785 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a8588e-8c26-4c9f-adc1-63a79a739195 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:27.527055 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e358568-fb74-42da-a140-536a4652d8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:27.527926 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e358568-fb74-42da-a140-536a4652d8fb None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:27.528175 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e358568-fb74-42da-a140-536a4652d8fb None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:27.528448 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e358568-fb74-42da-a140-536a4652d8fb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:27.550163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb558a-cb4f-42f4-a1f1-c43f9eeb85a1 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:27.553423 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e358568-fb74-42da-a140-536a4652d8fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:27.568477 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-5e358568-fb74-42da-a140-536a4652d8fb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 21:17:27.568477 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:27.568477 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:27.568477 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:27.568477 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:27.568477 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:27.568477 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:27.568477 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:27.568477 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:27.568477 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:27.568477 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:27.568477 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:27.568477 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:27.568477 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:27.568477 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:27.568477 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:27.569434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:27.569434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:27.569434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:27.569434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:27.569434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:27.569434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:27.569434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:27.569434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:27.569434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:27.569434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:27.569434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:27.569434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:27.569434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:27.569434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:27.569434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:27.569434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:27.570212 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:27.570212 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:27.570212 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:27.570212 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:27.570212 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:27.570212 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:27.570212 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:27.570212 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:27.570212 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:27.570212 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:27.570212 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 21:17:27.570212 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:27.573511 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 166/341] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:17:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:17:27.575774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394dee2-f593-4280-a1f3-c684ed8e41bd None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:27.576133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394dee2-f593-4280-a1f3-c684ed8e41bd None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:27.576487 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394dee2-f593-4280-a1f3-c684ed8e41bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:27.603989 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-7394dee2-f593-4280-a1f3-c684ed8e41bd None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:27.605451 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7394dee2-f593-4280-a1f3-c684ed8e41bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:27.610196 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f503b-79fb-4209-ada5-58ac6ded65a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:27.611224 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f503b-79fb-4209-ada5-58ac6ded65a0 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:27.611697 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f503b-79fb-4209-ada5-58ac6ded65a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:27.612229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f503b-79fb-4209-ada5-58ac6ded65a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:27.613259 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f503b-79fb-4209-ada5-58ac6ded65a0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:27.613857 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f503b-79fb-4209-ada5-58ac6ded65a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:27.615592 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f503b-79fb-4209-ada5-58ac6ded65a0 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:27.625881 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 176/342] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:17:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:17:27.627803 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472624d8-07a6-491d-962e-975ec250a44a None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:27.628404 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472624d8-07a6-491d-962e-975ec250a44a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:27.628881 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472624d8-07a6-491d-962e-975ec250a44a None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:27.630810 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 177/343] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:27.631743 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472624d8-07a6-491d-962e-975ec250a44a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:27.642793 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:27.697847 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 167/344] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:17:27.698851 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7394dee2-f593-4280-a1f3-c684ed8e41bd None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:27.762541 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:27.769963 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472624d8-07a6-491d-962e-975ec250a44a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:27.815362 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3458ce31-3098-4bed-8eaa-a3077df8b8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:27.816488 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458ce31-3098-4bed-8eaa-a3077df8b8aa None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:27.816981 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458ce31-3098-4bed-8eaa-a3077df8b8aa None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:27.817464 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458ce31-3098-4bed-8eaa-a3077df8b8aa None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:27.818464 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3458ce31-3098-4bed-8eaa-a3077df8b8aa None admin] RBAC: Adding query filter params (name=member) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:27.818974 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3458ce31-3098-4bed-8eaa-a3077df8b8aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:27.820655 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3458ce31-3098-4bed-8eaa-a3077df8b8aa None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:27.826396 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7394dee2-f593-4280-a1f3-c684ed8e41bd None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:27.831794 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 178/345] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 21:17:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:27.833475 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6085a15-53c7-4bb7-ba22-8e2f96684f7e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:27.833977 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6085a15-53c7-4bb7-ba22-8e2f96684f7e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:27.834472 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6085a15-53c7-4bb7-ba22-8e2f96684f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:27.866018 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d5966-b3b2-45fe-a665-4491f42c33b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:27.866694 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d6085a15-53c7-4bb7-ba22-8e2f96684f7e None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:27.866816 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d5966-b3b2-45fe-a665-4491f42c33b5 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:27.867013 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d5966-b3b2-45fe-a665-4491f42c33b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:27.867291 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d5966-b3b2-45fe-a665-4491f42c33b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:27.869243 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d6085a15-53c7-4bb7-ba22-8e2f96684f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:27.939382 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:27.948615 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 179/346] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:17:27.954399 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685d5966-b3b2-45fe-a665-4491f42c33b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:27.954399 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685d5966-b3b2-45fe-a665-4491f42c33b5 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:27.956088 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6085a15-53c7-4bb7-ba22-8e2f96684f7e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:28.021271 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:28.087393 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 168/347] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:17:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:17:28.089245 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d5966-b3b2-45fe-a665-4491f42c33b5 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:28.098558 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6085a15-53c7-4bb7-ba22-8e2f96684f7e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:28.140443 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44070289-820d-4be1-b90d-8994d4fb1687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:28.141293 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44070289-820d-4be1-b90d-8994d4fb1687 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:28.141571 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44070289-820d-4be1-b90d-8994d4fb1687 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:28.141844 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44070289-820d-4be1-b90d-8994d4fb1687 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:28.155242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44070289-820d-4be1-b90d-8994d4fb1687 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:28.163290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-44070289-820d-4be1-b90d-8994d4fb1687 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 21:17:28.163290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:28.163290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:28.163290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:28.163290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:28.163290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:28.163290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:28.163290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:28.163290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:28.163290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:28.163290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:28.163290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:28.163290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:28.163290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:28.163290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:28.163290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:28.164318 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:28.164318 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:28.164318 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:28.164318 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:28.164318 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:28.164318 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:28.164318 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:28.164318 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:28.164318 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:28.164318 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:28.164318 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:28.164318 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:17:28.164318 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:28.164318 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:28.164318 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:28.164318 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:28.165285 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:28.165285 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:28.165285 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:28.165285 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:28.165285 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:28.165285 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:28.165285 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:28.165285 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:28.165285 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:28.165285 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:28.165285 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 21:17:28.165285 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:17:28.165285 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 180/348] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:17:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:28.166693 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca2f3e-a996-493f-8977-5ae808c2b544 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:28.171219 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca2f3e-a996-493f-8977-5ae808c2b544 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:28.171511 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca2f3e-a996-493f-8977-5ae808c2b544 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:28.173285 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 181/349] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:28.174061 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca2f3e-a996-493f-8977-5ae808c2b544 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:28.221398 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d5966-b3b2-45fe-a665-4491f42c33b5 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:28.252543 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:28.274988 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac307df-2aad-4758-ab55-22daeed85665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:28.275714 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac307df-2aad-4758-ab55-22daeed85665 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:28.275992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac307df-2aad-4758-ab55-22daeed85665 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:28.276315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac307df-2aad-4758-ab55-22daeed85665 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:28.304835 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:28.309534 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca2f3e-a996-493f-8977-5ae808c2b544 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:28.356632 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00532eb9-aaf3-4577-8cf2-5e3591072f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:28.358053 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00532eb9-aaf3-4577-8cf2-5e3591072f6c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:28.358768 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00532eb9-aaf3-4577-8cf2-5e3591072f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:28.359422 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00532eb9-aaf3-4577-8cf2-5e3591072f6c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:28.368415 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac307df-2aad-4758-ab55-22daeed85665 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:28.370220 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac307df-2aad-4758-ab55-22daeed85665 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:28.376103 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00532eb9-aaf3-4577-8cf2-5e3591072f6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:28.384873 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-00532eb9-aaf3-4577-8cf2-5e3591072f6c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 21:17:28.384873 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:28.384873 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:28.384873 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:28.384873 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:28.384873 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:28.384873 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:28.384873 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:28.384873 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:28.384873 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:28.384873 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:28.384873 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:28.384873 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:28.384873 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:28.384873 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:28.384873 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:28.385865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:28.385865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:28.385865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:28.385865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:28.385865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:28.385865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:28.385865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:28.385865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:28.385865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:28.385865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:28.385865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:28.385865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:17:28.385865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:28.385865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:28.385865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:28.385865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:28.387070 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:28.387070 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:28.387070 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:28.387070 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:28.387070 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:28.387070 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:28.387070 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:28.387070 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:28.387070 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:28.387070 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:28.387070 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 21:17:28.387070 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:17:28.387070 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 182/350] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:17:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:28.387735 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00532eb9-aaf3-4577-8cf2-5e3591072f6c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:28.494176 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 169/351] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:17:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:17:28.495084 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac307df-2aad-4758-ab55-22daeed85665 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:28.514127 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00532eb9-aaf3-4577-8cf2-5e3591072f6c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:28.534696 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:28.562603 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:28.565181 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3dfccc-ece3-464d-b231-21faabf4e24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:28.566086 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3dfccc-ece3-464d-b231-21faabf4e24e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:28.566394 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3dfccc-ece3-464d-b231-21faabf4e24e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:28.566668 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3dfccc-ece3-464d-b231-21faabf4e24e None admin] PATH_INFO: `/v3/projects/c12a755e369e4c55a7d4a855fc16d986` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:28.570893 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3dfccc-ece3-464d-b231-21faabf4e24e None admin] RBAC: Authorizing `identity:get_project(project_id=c12a755e369e4c55a7d4a855fc16d986)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:28.572654 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3dfccc-ece3-464d-b231-21faabf4e24e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:28.574605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 183/352] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:17:28 2022] GET /identity/v3/projects/c12a755e369e4c55a7d4a855fc16d986 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:28.577084 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d848874-0ac2-40d7-9dfd-b02a95569c5d None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:28.577084 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d848874-0ac2-40d7-9dfd-b02a95569c5d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:28.577084 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d848874-0ac2-40d7-9dfd-b02a95569c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:28.594604 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-5d848874-0ac2-40d7-9dfd-b02a95569c5d None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:28.595815 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5d848874-0ac2-40d7-9dfd-b02a95569c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:28.633282 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac307df-2aad-4758-ab55-22daeed85665 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:28.680995 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 184/353] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:17:28.682071 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d848874-0ac2-40d7-9dfd-b02a95569c5d None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:28.685777 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95cbfbc-a762-4cfc-8bb9-f18678ae6557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:28.686588 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95cbfbc-a762-4cfc-8bb9-f18678ae6557 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:28.686852 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95cbfbc-a762-4cfc-8bb9-f18678ae6557 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:28.687128 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95cbfbc-a762-4cfc-8bb9-f18678ae6557 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:28.687893 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95cbfbc-a762-4cfc-8bb9-f18678ae6557 None admin] RBAC: Adding query filter params (name=member) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:28.688203 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95cbfbc-a762-4cfc-8bb9-f18678ae6557 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:28.689988 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95cbfbc-a762-4cfc-8bb9-f18678ae6557 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:28.699297 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 170/354] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 21:17:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:28.700098 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95cbfbc-a762-4cfc-8bb9-f18678ae6557 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:28.808351 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d848874-0ac2-40d7-9dfd-b02a95569c5d None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:28.824328 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:28.825090 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95cbfbc-a762-4cfc-8bb9-f18678ae6557 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:28.837903 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:28.853806 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94539b6a-d663-4bfb-8a2f-d5c4fbf146ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:28.854547 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94539b6a-d663-4bfb-8a2f-d5c4fbf146ca None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:28.854835 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94539b6a-d663-4bfb-8a2f-d5c4fbf146ca None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:28.855110 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94539b6a-d663-4bfb-8a2f-d5c4fbf146ca None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:28.855874 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94539b6a-d663-4bfb-8a2f-d5c4fbf146ca None admin] RBAC: Adding query filter params (name=reader) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:28.856180 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94539b6a-d663-4bfb-8a2f-d5c4fbf146ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:28.857570 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94539b6a-d663-4bfb-8a2f-d5c4fbf146ca None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:28.868002 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 185/355] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 21:17:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:28.869372 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27cfcbd-b60a-40ce-b575-de5bc00977da None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:28.869670 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27cfcbd-b60a-40ce-b575-de5bc00977da None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:28.869945 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27cfcbd-b60a-40ce-b575-de5bc00977da None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:28.870582 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f64158-83e7-4c14-9c9b-c302a1bc4e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:28.874366 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f64158-83e7-4c14-9c9b-c302a1bc4e4d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:28.874496 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 186/356] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:28.874591 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f64158-83e7-4c14-9c9b-c302a1bc4e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:28.874591 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f64158-83e7-4c14-9c9b-c302a1bc4e4d None admin] PATH_INFO: `/v3/projects/c12a755e369e4c55a7d4a855fc16d986` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:28.874739 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27cfcbd-b60a-40ce-b575-de5bc00977da None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:28.875607 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f64158-83e7-4c14-9c9b-c302a1bc4e4d None admin] RBAC: Authorizing `identity:get_project(project_id=c12a755e369e4c55a7d4a855fc16d986)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:28.877066 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f64158-83e7-4c14-9c9b-c302a1bc4e4d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:28.880252 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 171/357] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:17:28 2022] GET /identity/v3/projects/c12a755e369e4c55a7d4a855fc16d986 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:28.881438 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f64158-83e7-4c14-9c9b-c302a1bc4e4d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:29.003242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27cfcbd-b60a-40ce-b575-de5bc00977da None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:29.007421 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f64158-83e7-4c14-9c9b-c302a1bc4e4d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:29.046278 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f32cfe2-3508-49e2-8363-317ddacd480d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:29.047359 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32cfe2-3508-49e2-8363-317ddacd480d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:29.047649 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32cfe2-3508-49e2-8363-317ddacd480d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:29.048244 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32cfe2-3508-49e2-8363-317ddacd480d None admin] PATH_INFO: `/v3/groups/f0c359348291493c8f55343e5f9c5054` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:29.056648 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7c5996-78a7-47eb-b4a5-9f8246846af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:29.057357 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c5996-78a7-47eb-b4a5-9f8246846af1 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:29.057641 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c5996-78a7-47eb-b4a5-9f8246846af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:29.057739 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f32cfe2-3508-49e2-8363-317ddacd480d None admin] RBAC: Authorizing `identity:get_group(group_id=f0c359348291493c8f55343e5f9c5054)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:29.057913 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c5996-78a7-47eb-b4a5-9f8246846af1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:29.059322 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f32cfe2-3508-49e2-8363-317ddacd480d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:29.061414 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 187/358] 10.222.0.65 () {56 vars in 1180 bytes} [Thu May 19 21:17:28 2022] GET /identity/v3/groups/f0c359348291493c8f55343e5f9c5054 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:29.063270 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f32cfe2-3508-49e2-8363-317ddacd480d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:29.117963 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:29.125146 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:29.156666 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7c5996-78a7-47eb-b4a5-9f8246846af1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:29.158463 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7c5996-78a7-47eb-b4a5-9f8246846af1 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:29.218511 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f32cfe2-3508-49e2-8363-317ddacd480d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:29.267862 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d8c53-5eb2-4ecb-8561-dbb3d47d951c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:29.268730 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d8c53-5eb2-4ecb-8561-dbb3d47d951c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:29.269014 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d8c53-5eb2-4ecb-8561-dbb3d47d951c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:29.269288 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d8c53-5eb2-4ecb-8561-dbb3d47d951c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:29.280267 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 172/359] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:17:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:17:29.281487 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7c5996-78a7-47eb-b4a5-9f8246846af1 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:29.284354 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d8c53-5eb2-4ecb-8561-dbb3d47d951c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:29.295242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-910d8c53-5eb2-4ecb-8561-dbb3d47d951c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:17:29.295242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:29.295242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:29.295242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:29.295242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:29.295242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:29.295242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:29.295242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:29.295242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:29.295242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:29.295242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:29.295242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:29.295242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:29.295242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:29.295242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:29.295242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:29.296183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:29.296183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:29.296183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:29.296183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:29.296183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:29.296183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:29.296183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:29.296183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:29.296183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:29.296183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:29.296183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:29.296183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:17:29.296183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:29.296183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:29.296183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:29.296183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:29.296965 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:29.296965 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:29.296965 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:29.296965 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:29.296965 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:29.296965 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:29.296965 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:29.296965 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:29.296965 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:29.296965 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:29.296965 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:17:29.296965 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:17:29.296965 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 188/360] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 21:17:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:29.298184 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7bec6-ff09-42fd-b2ce-8cc055c0afce None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:29.298184 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7bec6-ff09-42fd-b2ce-8cc055c0afce None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:29.298369 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7bec6-ff09-42fd-b2ce-8cc055c0afce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:29.315931 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a7a7bec6-ff09-42fd-b2ce-8cc055c0afce None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:29.317126 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a7a7bec6-ff09-42fd-b2ce-8cc055c0afce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:29.373415 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:29.395289 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 189/361] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:17:29.396197 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7bec6-ff09-42fd-b2ce-8cc055c0afce None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:29.403553 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7c5996-78a7-47eb-b4a5-9f8246846af1 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:29.434587 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:29.459026 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6668367-e3a8-4452-a9ea-76c0cf94fa57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:29.460271 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6668367-e3a8-4452-a9ea-76c0cf94fa57 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:29.460698 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6668367-e3a8-4452-a9ea-76c0cf94fa57 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:29.460698 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6668367-e3a8-4452-a9ea-76c0cf94fa57 None admin] PATH_INFO: `/v3/users/199b117f033f464fb07b2e658eb28c46` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:29.462628 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6668367-e3a8-4452-a9ea-76c0cf94fa57 None admin] RBAC: Authorizing `identity:get_user(user_id=199b117f033f464fb07b2e658eb28c46)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:29.465158 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6668367-e3a8-4452-a9ea-76c0cf94fa57 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:29.467202 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 173/362] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:17:29 2022] GET /identity/v3/users/199b117f033f464fb07b2e658eb28c46 => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:29.467842 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6668367-e3a8-4452-a9ea-76c0cf94fa57 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:29.523338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7bec6-ff09-42fd-b2ce-8cc055c0afce None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:29.568495 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745cbe1-bc2f-4c94-81ea-bcf7c347677f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:29.569295 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745cbe1-bc2f-4c94-81ea-bcf7c347677f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:29.569575 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745cbe1-bc2f-4c94-81ea-bcf7c347677f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:29.569849 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745cbe1-bc2f-4c94-81ea-bcf7c347677f None admin] PATH_INFO: `/v3/groups/f0c359348291493c8f55343e5f9c5054` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:29.571194 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6745cbe1-bc2f-4c94-81ea-bcf7c347677f None admin] RBAC: Authorizing `identity:get_group(group_id=f0c359348291493c8f55343e5f9c5054)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:29.572665 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6745cbe1-bc2f-4c94-81ea-bcf7c347677f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:29.579052 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 190/363] 10.222.0.65 () {56 vars in 1180 bytes} [Thu May 19 21:17:29 2022] GET /identity/v3/groups/f0c359348291493c8f55343e5f9c5054 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:29.579052 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745cbe1-bc2f-4c94-81ea-bcf7c347677f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:29.583837 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6668367-e3a8-4452-a9ea-76c0cf94fa57 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:29.620650 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c2248-6a73-4c92-bf08-a96727ab4a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:29.621309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c2248-6a73-4c92-bf08-a96727ab4a1f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:29.621592 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c2248-6a73-4c92-bf08-a96727ab4a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:29.621868 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c2248-6a73-4c92-bf08-a96727ab4a1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:29.623032 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318c2248-6a73-4c92-bf08-a96727ab4a1f None admin] RBAC: Adding query filter params (group.id=f0c359348291493c8f55343e5f9c5054, role.id=566c421659204df185f8b77bf3bb3afc, scope.project.id=c12a755e369e4c55a7d4a855fc16d986) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:29.623032 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318c2248-6a73-4c92-bf08-a96727ab4a1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:29.624453 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318c2248-6a73-4c92-bf08-a96727ab4a1f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:29.630206 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 174/364] 10.222.0.65 () {56 vars in 1376 bytes} [Thu May 19 21:17:29 2022] GET /identity/v3/role_assignments?group.id=f0c359348291493c8f55343e5f9c5054&scope.project.id=c12a755e369e4c55a7d4a855fc16d986&role.id=566c421659204df185f8b77bf3bb3afc => generated 260 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:29.630878 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c2248-6a73-4c92-bf08-a96727ab4a1f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:29.657197 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:29.687924 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:29.694340 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745cbe1-bc2f-4c94-81ea-bcf7c347677f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:29.745231 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b06258-b792-44dd-91e9-91ed28001b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:29.746552 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b06258-b792-44dd-91e9-91ed28001b45 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:29.746846 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b06258-b792-44dd-91e9-91ed28001b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:29.747296 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b06258-b792-44dd-91e9-91ed28001b45 None admin] PATH_INFO: `/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:29.748730 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b06258-b792-44dd-91e9-91ed28001b45 None admin] RBAC: Authorizing `identity:get_project(project_id=1264ceb6aa114fbf81b3d48cd8874c8f)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:29.750349 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b06258-b792-44dd-91e9-91ed28001b45 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:29.752418 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 191/365] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:17:29 2022] GET /identity/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:29.753096 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b06258-b792-44dd-91e9-91ed28001b45 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:29.781417 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c2248-6a73-4c92-bf08-a96727ab4a1f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:29.828858 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53899542-7417-48b4-a500-f1713cb5acfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:29.829470 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53899542-7417-48b4-a500-f1713cb5acfc None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:29.829751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53899542-7417-48b4-a500-f1713cb5acfc None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:29.830023 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53899542-7417-48b4-a500-f1713cb5acfc None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:29.831387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53899542-7417-48b4-a500-f1713cb5acfc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:29.831387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53899542-7417-48b4-a500-f1713cb5acfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:29.832595 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53899542-7417-48b4-a500-f1713cb5acfc None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:29.842379 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 175/366] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:17:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:29.845454 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53899542-7417-48b4-a500-f1713cb5acfc None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:29.876528 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b06258-b792-44dd-91e9-91ed28001b45 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:29.919215 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bde6e1-5314-44e2-a160-618b52eeff2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:29.920032 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bde6e1-5314-44e2-a160-618b52eeff2c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:29.920322 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bde6e1-5314-44e2-a160-618b52eeff2c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:29.920598 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bde6e1-5314-44e2-a160-618b52eeff2c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:29.933320 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bde6e1-5314-44e2-a160-618b52eeff2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:29.948710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:29.949706 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-37bde6e1-5314-44e2-a160-618b52eeff2c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:17:29.949706 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:29.949706 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:29.949706 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:29.949706 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:29.949706 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:29.949706 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:29.949706 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:29.949706 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:29.949706 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:29.949706 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:29.949706 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:29.949706 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:29.949706 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:29.949706 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:29.949706 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:29.951152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:29.951152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:29.951152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:29.951152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:29.951152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:29.951152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:29.951152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:29.951152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:29.951152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:29.951152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:29.951152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:29.951152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:17:29.951152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:29.951152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:29.951152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:29.951152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:29.952533 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:29.952533 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:29.952533 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:29.952533 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:29.952533 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:29.952533 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:29.952533 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:29.952533 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:29.952533 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:29.952533 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:29.952533 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:17:29.952533 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:17:29.953498 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 192/367] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 21:17:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:29.954064 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bde6e1-5314-44e2-a160-618b52eeff2c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:29.989783 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:29.997227 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53899542-7417-48b4-a500-f1713cb5acfc None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:30.032466 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b80787-6d09-49c8-ae1b-be01013565d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:30.033203 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b80787-6d09-49c8-ae1b-be01013565d4 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:30.033519 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b80787-6d09-49c8-ae1b-be01013565d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:30.033803 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b80787-6d09-49c8-ae1b-be01013565d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:30.034675 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b80787-6d09-49c8-ae1b-be01013565d4 None admin] RBAC: Adding query filter params (role.id=34a7a66da3654831a115d77e40a989da, user.id=199b117f033f464fb07b2e658eb28c46, scope.system=all) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:30.034994 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b80787-6d09-49c8-ae1b-be01013565d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:30.036576 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b80787-6d09-49c8-ae1b-be01013565d4 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:30.062276 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 176/368] 10.222.0.65 () {56 vars in 1308 bytes} [Thu May 19 21:17:29 2022] GET /identity/v3/role_assignments?user.id=199b117f033f464fb07b2e658eb28c46&scope.system=all&role.id=34a7a66da3654831a115d77e40a989da => generated 510 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:30.062276 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b80787-6d09-49c8-ae1b-be01013565d4 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:30.089475 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bde6e1-5314-44e2-a160-618b52eeff2c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:30.147097 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e86c02-9957-441e-8c0c-a83e86da18f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:30.148757 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e86c02-9957-441e-8c0c-a83e86da18f5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:30.149135 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e86c02-9957-441e-8c0c-a83e86da18f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:30.149518 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e86c02-9957-441e-8c0c-a83e86da18f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:30.150459 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e86c02-9957-441e-8c0c-a83e86da18f5 None admin] RBAC: Adding query filter params (group.id=f0c359348291493c8f55343e5f9c5054, role.id=70699fb0eae7441999835fa3b50350b3, scope.project.id=c12a755e369e4c55a7d4a855fc16d986) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:30.150778 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e86c02-9957-441e-8c0c-a83e86da18f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:30.152345 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e86c02-9957-441e-8c0c-a83e86da18f5 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:30.163501 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 193/369] 10.222.0.65 () {56 vars in 1376 bytes} [Thu May 19 21:17:29 2022] GET /identity/v3/role_assignments?group.id=f0c359348291493c8f55343e5f9c5054&scope.project.id=c12a755e369e4c55a7d4a855fc16d986&role.id=70699fb0eae7441999835fa3b50350b3 => generated 260 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:30.165047 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e86c02-9957-441e-8c0c-a83e86da18f5 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:30.187534 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b80787-6d09-49c8-ae1b-be01013565d4 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:30.222751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:30.245955 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a8c191-60d6-47e4-9f2b-72418796a6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:30.246945 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8c191-60d6-47e4-9f2b-72418796a6d1 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:30.247227 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8c191-60d6-47e4-9f2b-72418796a6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:30.247503 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8c191-60d6-47e4-9f2b-72418796a6d1 None admin] PATH_INFO: `/v3/groups/f0c359348291493c8f55343e5f9c5054` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:30.252076 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a8c191-60d6-47e4-9f2b-72418796a6d1 None admin] RBAC: Authorizing `identity:get_group(group_id=f0c359348291493c8f55343e5f9c5054)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:30.253689 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a8c191-60d6-47e4-9f2b-72418796a6d1 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:30.260601 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 177/370] 10.222.0.65 () {56 vars in 1180 bytes} [Thu May 19 21:17:30 2022] GET /identity/v3/groups/f0c359348291493c8f55343e5f9c5054 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:30.261524 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a8c191-60d6-47e4-9f2b-72418796a6d1 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:30.330983 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:30.345500 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e86c02-9957-441e-8c0c-a83e86da18f5 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:30.409687 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be4b47f-c79b-47a0-ab65-25240e6bcb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:30.410512 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4b47f-c79b-47a0-ab65-25240e6bcb0c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:30.410836 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4b47f-c79b-47a0-ab65-25240e6bcb0c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:30.411118 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4b47f-c79b-47a0-ab65-25240e6bcb0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:30.448059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a8c191-60d6-47e4-9f2b-72418796a6d1 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:30.493747 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ecbaca-60cf-4f7f-bcb0-089a3b87881c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:30.495124 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ecbaca-60cf-4f7f-bcb0-089a3b87881c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:30.495413 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ecbaca-60cf-4f7f-bcb0-089a3b87881c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:30.495692 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ecbaca-60cf-4f7f-bcb0-089a3b87881c None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:30.496571 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ecbaca-60cf-4f7f-bcb0-089a3b87881c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:30.496882 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ecbaca-60cf-4f7f-bcb0-089a3b87881c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:30.498307 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ecbaca-60cf-4f7f-bcb0-089a3b87881c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:30.509325 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 178/371] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:17:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:30.509965 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ecbaca-60cf-4f7f-bcb0-089a3b87881c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:30.516825 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be4b47f-c79b-47a0-ab65-25240e6bcb0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:30.518659 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be4b47f-c79b-47a0-ab65-25240e6bcb0c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:30.583020 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:30.645096 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ecbaca-60cf-4f7f-bcb0-089a3b87881c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:30.650654 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 194/372] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:17:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:17:30.650954 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:30.652117 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be4b47f-c79b-47a0-ab65-25240e6bcb0c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:30.699491 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd8943-bd86-4335-b863-80c2b115a863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:30.700168 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd8943-bd86-4335-b863-80c2b115a863 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:30.700452 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd8943-bd86-4335-b863-80c2b115a863 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:30.700728 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd8943-bd86-4335-b863-80c2b115a863 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:30.701624 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd8943-bd86-4335-b863-80c2b115a863 None admin] RBAC: Adding query filter params (group.id=f0c359348291493c8f55343e5f9c5054, role.id=70699fb0eae7441999835fa3b50350b3, scope.project.id=1264ceb6aa114fbf81b3d48cd8874c8f) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:30.701943 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd8943-bd86-4335-b863-80c2b115a863 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:30.703488 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd8943-bd86-4335-b863-80c2b115a863 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:30.711633 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 179/373] 10.222.0.65 () {56 vars in 1376 bytes} [Thu May 19 21:17:30 2022] GET /identity/v3/role_assignments?group.id=f0c359348291493c8f55343e5f9c5054&scope.project.id=1264ceb6aa114fbf81b3d48cd8874c8f&role.id=70699fb0eae7441999835fa3b50350b3 => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:30.712582 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd8943-bd86-4335-b863-80c2b115a863 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:30.814922 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be4b47f-c79b-47a0-ab65-25240e6bcb0c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:30.901163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b4631-e795-4643-87ac-2b2ddda73561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:30.901733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b4631-e795-4643-87ac-2b2ddda73561 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:30.902020 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b4631-e795-4643-87ac-2b2ddda73561 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:30.902321 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b4631-e795-4643-87ac-2b2ddda73561 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:30.921934 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd8943-bd86-4335-b863-80c2b115a863 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:30.949743 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:30.962995 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f541c03-3932-4536-b6dc-5fbbf0390097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:30.964285 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f541c03-3932-4536-b6dc-5fbbf0390097 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:30.964773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f541c03-3932-4536-b6dc-5fbbf0390097 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:30.965247 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f541c03-3932-4536-b6dc-5fbbf0390097 None admin] PATH_INFO: `/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:30.967835 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f541c03-3932-4536-b6dc-5fbbf0390097 None admin] RBAC: Authorizing `identity:get_project(project_id=1264ceb6aa114fbf81b3d48cd8874c8f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:30.969540 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f541c03-3932-4536-b6dc-5fbbf0390097 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:30.971566 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 180/374] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:17:30 2022] GET /identity/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f => generated 302 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:30.979266 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f541c03-3932-4536-b6dc-5fbbf0390097 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:31.014667 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:31.019367 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2b4631-e795-4643-87ac-2b2ddda73561 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:31.020884 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2b4631-e795-4643-87ac-2b2ddda73561 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:31.107929 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f541c03-3932-4536-b6dc-5fbbf0390097 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:31.147347 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d9f63-8db1-4ced-a3f4-b8c02d7e01a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:31.148564 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d9f63-8db1-4ced-a3f4-b8c02d7e01a5 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:31.149068 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d9f63-8db1-4ced-a3f4-b8c02d7e01a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:31.149683 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d9f63-8db1-4ced-a3f4-b8c02d7e01a5 None admin] PATH_INFO: `/v3/groups/f0c359348291493c8f55343e5f9c5054` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:31.151225 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0d9f63-8db1-4ced-a3f4-b8c02d7e01a5 None admin] RBAC: Authorizing `identity:get_group(group_id=f0c359348291493c8f55343e5f9c5054)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:31.152878 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0d9f63-8db1-4ced-a3f4-b8c02d7e01a5 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:31.155100 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 181/375] 10.222.0.65 () {56 vars in 1180 bytes} [Thu May 19 21:17:30 2022] GET /identity/v3/groups/f0c359348291493c8f55343e5f9c5054 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:31.161116 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 195/376] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:17:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:17:31.164845 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d9f63-8db1-4ced-a3f4-b8c02d7e01a5 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:31.170321 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b4631-e795-4643-87ac-2b2ddda73561 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:31.257732 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:31.319340 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d9f63-8db1-4ced-a3f4-b8c02d7e01a5 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:31.355820 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:31.368150 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324db259-b654-42bd-b73f-2bdb78aeea5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:31.369199 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324db259-b654-42bd-b73f-2bdb78aeea5a None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:31.369703 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324db259-b654-42bd-b73f-2bdb78aeea5a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:31.370183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324db259-b654-42bd-b73f-2bdb78aeea5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:31.371266 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324db259-b654-42bd-b73f-2bdb78aeea5a None admin] RBAC: Adding query filter params (group.id=f0c359348291493c8f55343e5f9c5054, role.id=566c421659204df185f8b77bf3bb3afc, scope.project.id=1264ceb6aa114fbf81b3d48cd8874c8f) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:31.371796 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324db259-b654-42bd-b73f-2bdb78aeea5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:31.372957 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b4631-e795-4643-87ac-2b2ddda73561 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:31.373708 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324db259-b654-42bd-b73f-2bdb78aeea5a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:31.388740 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 182/377] 10.222.0.65 () {56 vars in 1376 bytes} [Thu May 19 21:17:31 2022] GET /identity/v3/role_assignments?group.id=f0c359348291493c8f55343e5f9c5054&scope.project.id=1264ceb6aa114fbf81b3d48cd8874c8f&role.id=566c421659204df185f8b77bf3bb3afc => generated 260 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:31.427867 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59078cc4-a8ab-4990-a7f7-a4aec2582fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:31.429116 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59078cc4-a8ab-4990-a7f7-a4aec2582fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:31.429631 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59078cc4-a8ab-4990-a7f7-a4aec2582fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:31.430193 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59078cc4-a8ab-4990-a7f7-a4aec2582fd3 None admin] PATH_INFO: `/v3/projects/0436212ebbfa4ccb91d5457faaa851ee` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:31.436287 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59078cc4-a8ab-4990-a7f7-a4aec2582fd3 None admin] RBAC: Authorizing `identity:get_project(project_id=0436212ebbfa4ccb91d5457faaa851ee)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:31.438050 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59078cc4-a8ab-4990-a7f7-a4aec2582fd3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:31.440545 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 196/378] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:17:31 2022] GET /identity/v3/projects/0436212ebbfa4ccb91d5457faaa851ee => generated 344 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:31.446207 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324db259-b654-42bd-b73f-2bdb78aeea5a None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:31.605355 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324db259-b654-42bd-b73f-2bdb78aeea5a None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:31.633090 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:31.644859 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122ded6-2749-4e64-aef3-c838dde6a876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:31.645781 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122ded6-2749-4e64-aef3-c838dde6a876 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:31.646064 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122ded6-2749-4e64-aef3-c838dde6a876 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:31.646367 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122ded6-2749-4e64-aef3-c838dde6a876 None admin] PATH_INFO: `/v3/groups/d4def6f7c4e547e9aa539ed5b7ba04ce` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:31.653007 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122ded6-2749-4e64-aef3-c838dde6a876 None admin] RBAC: Authorizing `identity:get_group(group_id=d4def6f7c4e547e9aa539ed5b7ba04ce)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:31.654522 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122ded6-2749-4e64-aef3-c838dde6a876 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:31.659459 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 183/379] 10.222.0.65 () {56 vars in 1180 bytes} [Thu May 19 21:17:31 2022] GET /identity/v3/groups/d4def6f7c4e547e9aa539ed5b7ba04ce => generated 229 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:31.662634 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59078cc4-a8ab-4990-a7f7-a4aec2582fd3 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:31.793677 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59078cc4-a8ab-4990-a7f7-a4aec2582fd3 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:31.848402 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4182edb-df2d-429f-9ec0-b17921f7b740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:31.854286 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4182edb-df2d-429f-9ec0-b17921f7b740 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:31.854286 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4182edb-df2d-429f-9ec0-b17921f7b740 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:31.854286 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4182edb-df2d-429f-9ec0-b17921f7b740 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:31.854286 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4182edb-df2d-429f-9ec0-b17921f7b740 None admin] RBAC: Adding query filter params (group.id=d4def6f7c4e547e9aa539ed5b7ba04ce, role.id=2b714f2e5c6c469f8fa037bda1ed16f6, scope.project.id=0436212ebbfa4ccb91d5457faaa851ee) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:31.854286 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4182edb-df2d-429f-9ec0-b17921f7b740 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:31.854286 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4182edb-df2d-429f-9ec0-b17921f7b740 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:31.863286 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 197/380] 10.222.0.65 () {56 vars in 1376 bytes} [Thu May 19 21:17:31 2022] GET /identity/v3/role_assignments?group.id=d4def6f7c4e547e9aa539ed5b7ba04ce&scope.project.id=0436212ebbfa4ccb91d5457faaa851ee&role.id=2b714f2e5c6c469f8fa037bda1ed16f6 => generated 260 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:32.387439 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3cce0a-51b9-4fdc-8e3d-698d6d06eb95 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:32.388004 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3cce0a-51b9-4fdc-8e3d-698d6d06eb95 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:32.388452 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3cce0a-51b9-4fdc-8e3d-698d6d06eb95 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:32.392106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 184/381] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:32.400382 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279503e9-0cf8-42bb-89e3-849b74795992 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:32.400883 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279503e9-0cf8-42bb-89e3-849b74795992 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:32.400987 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279503e9-0cf8-42bb-89e3-849b74795992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:32.427300 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-279503e9-0cf8-42bb-89e3-849b74795992 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:32.428533 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-279503e9-0cf8-42bb-89e3-849b74795992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:32.452859 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:32.528619 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 198/382] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:17:32.565338 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3cce0a-51b9-4fdc-8e3d-698d6d06eb95 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:32.741680 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3cce0a-51b9-4fdc-8e3d-698d6d06eb95 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:32.791309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59993e7b-cb5a-468d-9a8d-1e999514b739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:32.792158 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59993e7b-cb5a-468d-9a8d-1e999514b739 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:32.792434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59993e7b-cb5a-468d-9a8d-1e999514b739 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:32.792705 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59993e7b-cb5a-468d-9a8d-1e999514b739 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:32.803667 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59993e7b-cb5a-468d-9a8d-1e999514b739 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:32.811902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-59993e7b-cb5a-468d-9a8d-1e999514b739 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:17:32.811902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:32.811902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:32.811902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:32.811902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:32.811902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:32.811902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:32.811902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:32.811902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:32.811902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:32.811902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:32.811902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:32.811902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:32.811902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:32.811902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:32.811902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:32.812832 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:32.812832 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:32.812832 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:32.812832 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:32.812832 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:32.812832 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:32.812832 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:32.812832 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:32.812832 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:32.812832 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:32.812832 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:32.812832 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:32.812832 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:32.812832 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:32.812832 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:32.812832 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:32.814070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:32.814070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:32.814070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:32.814070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:32.814070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:32.814070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:32.814070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:32.814070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:32.814070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:32.814070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:32.814070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:17:32.814070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:32.814070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 185/383] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 21:17:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:32.820918 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279503e9-0cf8-42bb-89e3-849b74795992 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:32.962361 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279503e9-0cf8-42bb-89e3-849b74795992 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:33.015217 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97750ddd-1959-4d58-ad4e-e6e0f26ccdf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:33.015828 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97750ddd-1959-4d58-ad4e-e6e0f26ccdf9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:33.016112 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97750ddd-1959-4d58-ad4e-e6e0f26ccdf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:33.016384 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97750ddd-1959-4d58-ad4e-e6e0f26ccdf9 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:33.017147 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97750ddd-1959-4d58-ad4e-e6e0f26ccdf9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:33.017456 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97750ddd-1959-4d58-ad4e-e6e0f26ccdf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:33.018888 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97750ddd-1959-4d58-ad4e-e6e0f26ccdf9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:33.029786 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 199/384] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:17:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:33.037043 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59993e7b-cb5a-468d-9a8d-1e999514b739 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:33.083012 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1af05-9680-4aa4-909b-d9056d1069a4 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:33.083643 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1af05-9680-4aa4-909b-d9056d1069a4 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:33.084176 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1af05-9680-4aa4-909b-d9056d1069a4 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:33.086063 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 200/385] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:33.091837 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d5fc1-8adb-4248-bdff-353bd447ba8d None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:33.092432 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d5fc1-8adb-4248-bdff-353bd447ba8d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:33.092908 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d5fc1-8adb-4248-bdff-353bd447ba8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:33.119377 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f32d5fc1-8adb-4248-bdff-353bd447ba8d None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:33.123769 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f32d5fc1-8adb-4248-bdff-353bd447ba8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:33.138698 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:33.145811 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:33.243264 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59993e7b-cb5a-468d-9a8d-1e999514b739 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:33.269844 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 201/386] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:17:33.304777 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b1b0d-2ec6-4700-b4d2-93d40a62b690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:33.305695 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b1b0d-2ec6-4700-b4d2-93d40a62b690 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:33.305980 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b1b0d-2ec6-4700-b4d2-93d40a62b690 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:33.306288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b1b0d-2ec6-4700-b4d2-93d40a62b690 None admin] PATH_INFO: `/v3/groups/f0c359348291493c8f55343e5f9c5054` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:33.306877 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32d5fc1-8adb-4248-bdff-353bd447ba8d None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:33.307504 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423b1b0d-2ec6-4700-b4d2-93d40a62b690 None admin] RBAC: Authorizing `identity:get_group(group_id=f0c359348291493c8f55343e5f9c5054)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:33.309028 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423b1b0d-2ec6-4700-b4d2-93d40a62b690 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:33.318297 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 186/387] 10.222.0.65 () {56 vars in 1180 bytes} [Thu May 19 21:17:33 2022] GET /identity/v3/groups/f0c359348291493c8f55343e5f9c5054 => generated 226 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:33.320212 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b1b0d-2ec6-4700-b4d2-93d40a62b690 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:33.416668 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32d5fc1-8adb-4248-bdff-353bd447ba8d None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:33.436012 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b1b0d-2ec6-4700-b4d2-93d40a62b690 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:33.453316 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a337a9f-c675-4994-bc46-9b348d5946f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:33.454125 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a337a9f-c675-4994-bc46-9b348d5946f8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:33.454432 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a337a9f-c675-4994-bc46-9b348d5946f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:33.454704 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a337a9f-c675-4994-bc46-9b348d5946f8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:33.462615 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:33.465528 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a337a9f-c675-4994-bc46-9b348d5946f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:33.473229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-3a337a9f-c675-4994-bc46-9b348d5946f8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 21:17:33.473229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:33.473229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:33.473229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:33.473229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:33.473229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:33.473229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:33.473229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:33.473229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:33.473229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:33.473229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:33.473229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:33.473229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:33.473229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:33.473229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:33.473229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:33.474185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:33.474185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:33.474185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:33.474185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:33.474185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:33.474185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:33.474185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:33.474185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:33.474185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:33.474185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:33.474185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:33.474185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:17:33.474185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:33.474185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:33.474185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:33.474185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:33.479078 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:33.479078 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:33.479078 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:33.479078 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:33.479078 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:33.479078 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:33.479078 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:33.479078 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:33.479078 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:33.479078 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:33.479078 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 21:17:33.479078 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:17:33.479078 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 202/388] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:17:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:33.479078 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67adbd-dfd9-42a8-ad2d-7aa8ffe2f5f7 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:33.480097 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67adbd-dfd9-42a8-ad2d-7aa8ffe2f5f7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:33.480097 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67adbd-dfd9-42a8-ad2d-7aa8ffe2f5f7 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:33.480097 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 203/389] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:33.483812 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c791d0f2-3949-48e7-bbad-000a9709cde3 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:33.484100 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c791d0f2-3949-48e7-bbad-000a9709cde3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:33.484378 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c791d0f2-3949-48e7-bbad-000a9709cde3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:33.486444 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f512387-b2da-4f36-8fd8-966622b35492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:33.487613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f512387-b2da-4f36-8fd8-966622b35492 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:33.488194 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f512387-b2da-4f36-8fd8-966622b35492 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:33.488642 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f512387-b2da-4f36-8fd8-966622b35492 None admin] PATH_INFO: `/v3/projects/c12a755e369e4c55a7d4a855fc16d986` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:33.490296 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f512387-b2da-4f36-8fd8-966622b35492 None admin] RBAC: Authorizing `identity:get_project(project_id=c12a755e369e4c55a7d4a855fc16d986)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:33.490839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:33.494178 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f512387-b2da-4f36-8fd8-966622b35492 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:33.502075 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 187/390] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:17:33 2022] GET /identity/v3/projects/c12a755e369e4c55a7d4a855fc16d986 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:33.503849 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f512387-b2da-4f36-8fd8-966622b35492 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:33.503959 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c791d0f2-3949-48e7-bbad-000a9709cde3 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:33.505373 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c791d0f2-3949-48e7-bbad-000a9709cde3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:33.577795 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 204/391] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:17:33.586269 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c791d0f2-3949-48e7-bbad-000a9709cde3 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:33.614863 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f512387-b2da-4f36-8fd8-966622b35492 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:33.655247 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ae35f-5fa8-4178-84f2-f4843904df35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:33.656426 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ae35f-5fa8-4178-84f2-f4843904df35 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:33.656909 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ae35f-5fa8-4178-84f2-f4843904df35 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:33.657445 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ae35f-5fa8-4178-84f2-f4843904df35 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:33.658419 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7ae35f-5fa8-4178-84f2-f4843904df35 None admin] RBAC: Adding query filter params (name=member) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:33.659030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7ae35f-5fa8-4178-84f2-f4843904df35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:33.660718 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7ae35f-5fa8-4178-84f2-f4843904df35 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:33.670716 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 188/392] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 21:17:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:33.675437 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ae35f-5fa8-4178-84f2-f4843904df35 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:33.705200 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c791d0f2-3949-48e7-bbad-000a9709cde3 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:33.729046 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:33.741279 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d92fe81-9f01-46de-bc96-a59d56b03298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:33.742263 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d92fe81-9f01-46de-bc96-a59d56b03298 None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:33.743082 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d92fe81-9f01-46de-bc96-a59d56b03298 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:33.743387 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d92fe81-9f01-46de-bc96-a59d56b03298 None admin] PATH_INFO: `/v3/projects/c12a755e369e4c55a7d4a855fc16d986/groups/f0c359348291493c8f55343e5f9c5054/roles/566c421659204df185f8b77bf3bb3afc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:33.745757 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d92fe81-9f01-46de-bc96-a59d56b03298 None admin] RBAC: Authorizing `identity:create_grant(project_id=c12a755e369e4c55a7d4a855fc16d986, group_id=f0c359348291493c8f55343e5f9c5054, role_id=566c421659204df185f8b77bf3bb3afc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:33.747316 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d92fe81-9f01-46de-bc96-a59d56b03298 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:33.749731 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:33.774271 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 205/393] 10.222.0.65 () {58 vars in 1442 bytes} [Thu May 19 21:17:33 2022] PUT /identity/v3/projects/c12a755e369e4c55a7d4a855fc16d986/groups/f0c359348291493c8f55343e5f9c5054/roles/566c421659204df185f8b77bf3bb3afc => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:17:33.774271 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d92fe81-9f01-46de-bc96-a59d56b03298 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:33.815853 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ae35f-5fa8-4178-84f2-f4843904df35 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:33.872961 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b60f8a-1c53-4803-b914-e183b049f2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:33.875371 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b60f8a-1c53-4803-b914-e183b049f2af None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:33.876171 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b60f8a-1c53-4803-b914-e183b049f2af None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:33.876665 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b60f8a-1c53-4803-b914-e183b049f2af None admin] PATH_INFO: `/v3/roles/member` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:33.902957 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b60f8a-1c53-4803-b914-e183b049f2af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:33.919905 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-d0b60f8a-1c53-4803-b914-e183b049f2af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 21:17:33.919905 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:33.919905 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:33.919905 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:33.919905 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:33.919905 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:33.919905 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:33.919905 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:33.919905 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:33.919905 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:33.919905 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:33.919905 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:33.919905 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:33.919905 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:33.919905 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:33.919905 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:33.920904 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:33.920904 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:33.920904 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:33.920904 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:33.920904 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:33.920904 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:33.920904 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:33.920904 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:33.920904 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:33.920904 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:33.920904 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:33.920904 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:33.920904 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:33.920904 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:33.920904 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:33.920904 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:33.921811 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:33.921811 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:33.921811 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:33.921811 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:33.921811 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:33.921811 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:33.921811 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:33.921811 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:33.921811 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:33.921811 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:33.921811 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 21:17:33.921811 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:33.924607 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 189/394] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:17:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:33.938872 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b60f8a-1c53-4803-b914-e183b049f2af None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:33.987424 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d92fe81-9f01-46de-bc96-a59d56b03298 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:34.032395 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b4761-33b3-4c2c-a793-1c1149244a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:34.033732 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b4761-33b3-4c2c-a793-1c1149244a01 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:34.034208 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b4761-33b3-4c2c-a793-1c1149244a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:34.034665 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b4761-33b3-4c2c-a793-1c1149244a01 None admin] PATH_INFO: `/v3/groups/f0c359348291493c8f55343e5f9c5054` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:34.037317 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b4761-33b3-4c2c-a793-1c1149244a01 None admin] RBAC: Authorizing `identity:get_group(group_id=f0c359348291493c8f55343e5f9c5054)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:34.039063 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b4761-33b3-4c2c-a793-1c1149244a01 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:34.041411 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 206/395] 10.222.0.65 () {56 vars in 1180 bytes} [Thu May 19 21:17:33 2022] GET /identity/v3/groups/f0c359348291493c8f55343e5f9c5054 => generated 226 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:34.047403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b4761-33b3-4c2c-a793-1c1149244a01 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:34.112750 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b60f8a-1c53-4803-b914-e183b049f2af None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:34.120914 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:34.123755 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:34.150423 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225751fa-77a5-4377-a33a-12eb69d46b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:34.151446 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225751fa-77a5-4377-a33a-12eb69d46b2b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:34.151906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225751fa-77a5-4377-a33a-12eb69d46b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:34.152350 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225751fa-77a5-4377-a33a-12eb69d46b2b None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:34.153271 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225751fa-77a5-4377-a33a-12eb69d46b2b None admin] RBAC: Adding query filter params (name=member) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:34.153751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225751fa-77a5-4377-a33a-12eb69d46b2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:34.155346 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225751fa-77a5-4377-a33a-12eb69d46b2b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:34.164682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 190/396] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 21:17:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:17:34.175525 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b4761-33b3-4c2c-a793-1c1149244a01 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:34.185290 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225751fa-77a5-4377-a33a-12eb69d46b2b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:34.213923 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1fe610-1755-4130-9bdc-00d84cdf8ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:34.214933 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fe610-1755-4130-9bdc-00d84cdf8ebb None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:34.215222 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fe610-1755-4130-9bdc-00d84cdf8ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:34.215500 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fe610-1755-4130-9bdc-00d84cdf8ebb None admin] PATH_INFO: `/v3/projects/c12a755e369e4c55a7d4a855fc16d986` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:34.216724 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1fe610-1755-4130-9bdc-00d84cdf8ebb None admin] RBAC: Authorizing `identity:get_project(project_id=c12a755e369e4c55a7d4a855fc16d986)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:34.218202 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1fe610-1755-4130-9bdc-00d84cdf8ebb None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:34.224505 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 207/397] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:17:34 2022] GET /identity/v3/projects/c12a755e369e4c55a7d4a855fc16d986 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:34.226365 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1fe610-1755-4130-9bdc-00d84cdf8ebb None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:34.328014 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225751fa-77a5-4377-a33a-12eb69d46b2b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:34.343089 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1fe610-1755-4130-9bdc-00d84cdf8ebb None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:34.365253 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6446a2-3009-4c60-ab81-1f1a3c0722b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:34.366151 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6446a2-3009-4c60-ab81-1f1a3c0722b9 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:34.366554 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6446a2-3009-4c60-ab81-1f1a3c0722b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:34.366894 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6446a2-3009-4c60-ab81-1f1a3c0722b9 None admin] PATH_INFO: `/v3/groups/f0c359348291493c8f55343e5f9c5054` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:34.368577 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6446a2-3009-4c60-ab81-1f1a3c0722b9 None admin] RBAC: Authorizing `identity:get_group(group_id=f0c359348291493c8f55343e5f9c5054)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:34.370100 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6446a2-3009-4c60-ab81-1f1a3c0722b9 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:34.372146 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 191/398] 10.222.0.65 () {56 vars in 1180 bytes} [Thu May 19 21:17:34 2022] GET /identity/v3/groups/f0c359348291493c8f55343e5f9c5054 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:17:34.373461 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e3918-6fdb-4724-8e98-807a1b2450df None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:34.373819 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e3918-6fdb-4724-8e98-807a1b2450df None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:34.374162 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e3918-6fdb-4724-8e98-807a1b2450df None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:34.375760 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 192/399] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:34.381066 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef47c14-cacc-4146-a8b7-72acf3918294 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:34.381354 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef47c14-cacc-4146-a8b7-72acf3918294 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:34.381635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef47c14-cacc-4146-a8b7-72acf3918294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:34.386297 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:34.389149 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b95f152-a445-49a0-87cb-2162fc9fe7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:34.390075 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b95f152-a445-49a0-87cb-2162fc9fe7dc None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:34.390415 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b95f152-a445-49a0-87cb-2162fc9fe7dc None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:34.390731 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b95f152-a445-49a0-87cb-2162fc9fe7dc None admin] PATH_INFO: `/v3/projects/c12a755e369e4c55a7d4a855fc16d986/groups/f0c359348291493c8f55343e5f9c5054/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:34.393767 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b95f152-a445-49a0-87cb-2162fc9fe7dc None admin] RBAC: Authorizing `identity:create_grant(project_id=c12a755e369e4c55a7d4a855fc16d986, group_id=f0c359348291493c8f55343e5f9c5054, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:34.395482 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b95f152-a445-49a0-87cb-2162fc9fe7dc None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:34.401730 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-1ef47c14-cacc-4146-a8b7-72acf3918294 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:34.403379 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1ef47c14-cacc-4146-a8b7-72acf3918294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:34.414375 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 208/400] 10.222.0.65 () {58 vars in 1442 bytes} [Thu May 19 21:17:34 2022] PUT /identity/v3/projects/c12a755e369e4c55a7d4a855fc16d986/groups/f0c359348291493c8f55343e5f9c5054/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:17:34.415811 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b95f152-a445-49a0-87cb-2162fc9fe7dc None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:34.464531 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:34.540263 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 193/401] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:17:34.584570 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef47c14-cacc-4146-a8b7-72acf3918294 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:34.619268 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b95f152-a445-49a0-87cb-2162fc9fe7dc None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:34.665977 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04987ccd-c2b8-4871-82ba-9afaa9bd44c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:34.667259 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04987ccd-c2b8-4871-82ba-9afaa9bd44c2 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:34.668049 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04987ccd-c2b8-4871-82ba-9afaa9bd44c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:34.668266 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04987ccd-c2b8-4871-82ba-9afaa9bd44c2 None admin] PATH_INFO: `/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:34.670328 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04987ccd-c2b8-4871-82ba-9afaa9bd44c2 None admin] RBAC: Authorizing `identity:get_project(project_id=1264ceb6aa114fbf81b3d48cd8874c8f)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:34.671780 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04987ccd-c2b8-4871-82ba-9afaa9bd44c2 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:34.673673 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 209/402] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:17:34 2022] GET /identity/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f => generated 302 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:34.679170 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04987ccd-c2b8-4871-82ba-9afaa9bd44c2 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:34.743179 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef47c14-cacc-4146-a8b7-72acf3918294 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:34.746600 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:34.782617 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3af0a8-e6fc-4c29-b444-999902f78796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:34.784064 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3af0a8-e6fc-4c29-b444-999902f78796 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:34.784458 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3af0a8-e6fc-4c29-b444-999902f78796 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:34.784836 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3af0a8-e6fc-4c29-b444-999902f78796 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:34.796951 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04987ccd-c2b8-4871-82ba-9afaa9bd44c2 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:34.799278 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3af0a8-e6fc-4c29-b444-999902f78796 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:34.810081 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-9f3af0a8-e6fc-4c29-b444-999902f78796 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:17:34.810081 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:34.810081 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:34.810081 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:34.810081 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:34.810081 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:34.810081 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:34.810081 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:34.810081 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:34.810081 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:34.810081 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:34.810081 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:34.810081 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:34.810081 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:34.810081 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:34.810081 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:34.811118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:34.811118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:34.811118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:34.811118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:34.811118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:34.811118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:34.811118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:34.811118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:34.811118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:34.811118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:34.811118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:34.811118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:34.811118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:34.811118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:34.811118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:34.811118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:34.811921 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:34.811921 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:34.811921 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:34.811921 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:34.811921 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:34.811921 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:34.811921 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:34.811921 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:34.811921 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:34.811921 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:34.811921 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:17:34.811921 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:34.812547 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 194/403] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 21:17:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:34.814000 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930459a2-1c6a-492d-a7d9-38e9fc6e21d5 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:34.814390 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930459a2-1c6a-492d-a7d9-38e9fc6e21d5 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:34.814915 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930459a2-1c6a-492d-a7d9-38e9fc6e21d5 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:34.816934 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 195/404] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:34.821406 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930459a2-1c6a-492d-a7d9-38e9fc6e21d5 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:34.846962 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:34.853592 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6da751-752b-44ad-a194-7f185823b3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:34.854781 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6da751-752b-44ad-a194-7f185823b3a2 None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:34.855133 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6da751-752b-44ad-a194-7f185823b3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:34.855479 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6da751-752b-44ad-a194-7f185823b3a2 None admin] PATH_INFO: `/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f/groups/f0c359348291493c8f55343e5f9c5054/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:34.860868 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6da751-752b-44ad-a194-7f185823b3a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=1264ceb6aa114fbf81b3d48cd8874c8f, group_id=f0c359348291493c8f55343e5f9c5054, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:34.862377 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6da751-752b-44ad-a194-7f185823b3a2 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:34.881599 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 210/405] 10.222.0.65 () {58 vars in 1442 bytes} [Thu May 19 21:17:34 2022] PUT /identity/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f/groups/f0c359348291493c8f55343e5f9c5054/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:17:34.883300 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6be51-bb92-4afa-b0ae-bd2a5b145c5b None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:34.883643 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6be51-bb92-4afa-b0ae-bd2a5b145c5b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:34.883927 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6be51-bb92-4afa-b0ae-bd2a5b145c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:34.906307 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-bae6be51-bb92-4afa-b0ae-bd2a5b145c5b None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:34.907555 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-bae6be51-bb92-4afa-b0ae-bd2a5b145c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:34.982135 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930459a2-1c6a-492d-a7d9-38e9fc6e21d5 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:35.037924 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 211/406] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:17:35.046634 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded87e8-2bbc-406a-9601-59bf01cbdaf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:35.047314 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded87e8-2bbc-406a-9601-59bf01cbdaf6 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:35.047599 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded87e8-2bbc-406a-9601-59bf01cbdaf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:35.047873 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded87e8-2bbc-406a-9601-59bf01cbdaf6 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:35.048681 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded87e8-2bbc-406a-9601-59bf01cbdaf6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:35.048994 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded87e8-2bbc-406a-9601-59bf01cbdaf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:35.050426 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded87e8-2bbc-406a-9601-59bf01cbdaf6 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:35.060085 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 196/407] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:17:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:35.068848 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae6be51-bb92-4afa-b0ae-bd2a5b145c5b None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:35.075900 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded87e8-2bbc-406a-9601-59bf01cbdaf6 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:35.165215 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:35.187743 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:35.202478 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded87e8-2bbc-406a-9601-59bf01cbdaf6 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:35.243161 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae6be51-bb92-4afa-b0ae-bd2a5b145c5b None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:35.250928 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5858a8f-a934-4b47-a147-3d8491523335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:35.252287 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5858a8f-a934-4b47-a147-3d8491523335 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:35.252578 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5858a8f-a934-4b47-a147-3d8491523335 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:35.252860 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5858a8f-a934-4b47-a147-3d8491523335 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:35.268105 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5858a8f-a934-4b47-a147-3d8491523335 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:35.276647 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-a5858a8f-a934-4b47-a147-3d8491523335 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:17:35.276647 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:35.276647 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:35.276647 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:35.276647 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:35.276647 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:35.276647 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:35.276647 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:35.276647 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:35.276647 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:35.276647 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:35.276647 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:35.276647 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:35.276647 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:35.276647 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:35.276647 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:35.277546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:35.277546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:35.277546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:35.277546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:35.277546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:35.277546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:35.277546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:35.277546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:35.277546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:35.277546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:35.277546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:35.277546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:35.277546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:35.277546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:35.277546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:35.277546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:35.278465 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:35.278465 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:35.278465 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:35.278465 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:35.278465 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:35.278465 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:35.278465 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:35.278465 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:35.278465 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:35.278465 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:35.278465 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:17:35.278465 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:35.280848 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 197/408] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 21:17:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:17:35.287811 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5858a8f-a934-4b47-a147-3d8491523335 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:35.288199 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb1d82a-3398-42ab-9677-8036ba773dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:35.289268 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1d82a-3398-42ab-9677-8036ba773dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:35.289559 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1d82a-3398-42ab-9677-8036ba773dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:35.289842 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1d82a-3398-42ab-9677-8036ba773dd3 None admin] PATH_INFO: `/v3/groups/f0c359348291493c8f55343e5f9c5054` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:35.291157 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb1d82a-3398-42ab-9677-8036ba773dd3 None admin] RBAC: Authorizing `identity:get_group(group_id=f0c359348291493c8f55343e5f9c5054)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:35.292665 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb1d82a-3398-42ab-9677-8036ba773dd3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:35.298273 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 212/409] 10.222.0.65 () {56 vars in 1180 bytes} [Thu May 19 21:17:35 2022] GET /identity/v3/groups/f0c359348291493c8f55343e5f9c5054 => generated 226 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:35.300815 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb1d82a-3398-42ab-9677-8036ba773dd3 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:35.427443 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb1d82a-3398-42ab-9677-8036ba773dd3 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:35.470714 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:35.496654 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8172d127-38f4-4938-9242-ac11cd992071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:35.497533 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172d127-38f4-4938-9242-ac11cd992071 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:35.497815 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172d127-38f4-4938-9242-ac11cd992071 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:35.498091 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172d127-38f4-4938-9242-ac11cd992071 None admin] PATH_INFO: `/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:35.499431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8172d127-38f4-4938-9242-ac11cd992071 None admin] RBAC: Authorizing `identity:get_project(project_id=1264ceb6aa114fbf81b3d48cd8874c8f)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:35.501773 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8172d127-38f4-4938-9242-ac11cd992071 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:35.504790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 213/410] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:17:35 2022] GET /identity/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:35.506712 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5858a8f-a934-4b47-a147-3d8491523335 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:35.509573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8172d127-38f4-4938-9242-ac11cd992071 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:35.573687 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a62e15-69a4-4622-add5-94191c3cb3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:35.574396 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a62e15-69a4-4622-add5-94191c3cb3df None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:35.574684 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a62e15-69a4-4622-add5-94191c3cb3df None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:35.574957 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a62e15-69a4-4622-add5-94191c3cb3df None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:35.575712 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a62e15-69a4-4622-add5-94191c3cb3df None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:35.576023 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a62e15-69a4-4622-add5-94191c3cb3df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:35.577413 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a62e15-69a4-4622-add5-94191c3cb3df None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:35.603887 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 198/411] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:17:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:35.606759 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a62e15-69a4-4622-add5-94191c3cb3df None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:35.630601 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:35.716399 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8172d127-38f4-4938-9242-ac11cd992071 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:35.790994 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de19bf-1f1d-49e3-982f-6f79558b44af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:35.791896 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de19bf-1f1d-49e3-982f-6f79558b44af None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:35.792180 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de19bf-1f1d-49e3-982f-6f79558b44af None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:35.792456 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de19bf-1f1d-49e3-982f-6f79558b44af None admin] PATH_INFO: `/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f/groups/f0c359348291493c8f55343e5f9c5054/roles/566c421659204df185f8b77bf3bb3afc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:35.796256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86de19bf-1f1d-49e3-982f-6f79558b44af None admin] RBAC: Authorizing `identity:create_grant(project_id=1264ceb6aa114fbf81b3d48cd8874c8f, group_id=f0c359348291493c8f55343e5f9c5054, role_id=566c421659204df185f8b77bf3bb3afc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:35.797780 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86de19bf-1f1d-49e3-982f-6f79558b44af None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:35.811247 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a62e15-69a4-4622-add5-94191c3cb3df None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:35.825964 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 214/412] 10.222.0.65 () {58 vars in 1442 bytes} [Thu May 19 21:17:35 2022] PUT /identity/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f/groups/f0c359348291493c8f55343e5f9c5054/roles/566c421659204df185f8b77bf3bb3afc => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:17:35.883576 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3532790b-89c6-41a4-9044-6c1d8c10c96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:35.884503 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3532790b-89c6-41a4-9044-6c1d8c10c96c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:35.884790 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3532790b-89c6-41a4-9044-6c1d8c10c96c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:35.885069 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3532790b-89c6-41a4-9044-6c1d8c10c96c None admin] PATH_INFO: `/v3/groups/d4def6f7c4e547e9aa539ed5b7ba04ce` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:35.886524 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3532790b-89c6-41a4-9044-6c1d8c10c96c None admin] RBAC: Authorizing `identity:get_group(group_id=d4def6f7c4e547e9aa539ed5b7ba04ce)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:35.888031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3532790b-89c6-41a4-9044-6c1d8c10c96c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:35.889810 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 199/413] 10.222.0.65 () {56 vars in 1180 bytes} [Thu May 19 21:17:35 2022] GET /identity/v3/groups/d4def6f7c4e547e9aa539ed5b7ba04ce => generated 229 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:17:35.895668 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de19bf-1f1d-49e3-982f-6f79558b44af None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:35.996696 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:36.074099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de19bf-1f1d-49e3-982f-6f79558b44af None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:36.117937 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a16210-1f65-42bf-b719-43fa2360d084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:36.119860 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a16210-1f65-42bf-b719-43fa2360d084 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:36.120619 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a16210-1f65-42bf-b719-43fa2360d084 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:36.121371 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a16210-1f65-42bf-b719-43fa2360d084 None admin] PATH_INFO: `/v3/projects/0436212ebbfa4ccb91d5457faaa851ee` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:36.123811 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a16210-1f65-42bf-b719-43fa2360d084 None admin] RBAC: Authorizing `identity:get_project(project_id=0436212ebbfa4ccb91d5457faaa851ee)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:36.126093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a16210-1f65-42bf-b719-43fa2360d084 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:36.128647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 215/414] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:17:35 2022] GET /identity/v3/projects/0436212ebbfa4ccb91d5457faaa851ee => generated 344 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:36.139412 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3532790b-89c6-41a4-9044-6c1d8c10c96c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:36.266848 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:36.297373 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3532790b-89c6-41a4-9044-6c1d8c10c96c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:36.362099 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684e7012-4c19-4132-a9bb-c2e4478c766a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:36.363283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e7012-4c19-4132-a9bb-c2e4478c766a None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:36.363708 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e7012-4c19-4132-a9bb-c2e4478c766a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:36.364117 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e7012-4c19-4132-a9bb-c2e4478c766a None admin] PATH_INFO: `/v3/projects/0436212ebbfa4ccb91d5457faaa851ee/groups/d4def6f7c4e547e9aa539ed5b7ba04ce/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:36.368482 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684e7012-4c19-4132-a9bb-c2e4478c766a None admin] RBAC: Authorizing `identity:create_grant(project_id=0436212ebbfa4ccb91d5457faaa851ee, group_id=d4def6f7c4e547e9aa539ed5b7ba04ce, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:36.369968 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684e7012-4c19-4132-a9bb-c2e4478c766a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:36.397780 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 200/415] 10.222.0.65 () {58 vars in 1442 bytes} [Thu May 19 21:17:36 2022] PUT /identity/v3/projects/0436212ebbfa4ccb91d5457faaa851ee/groups/d4def6f7c4e547e9aa539ed5b7ba04ce/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:17:36.512098 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1b9d7-15df-4c84-b580-e6fb1e5e4937 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:36.512608 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1b9d7-15df-4c84-b580-e6fb1e5e4937 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:36.512983 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1b9d7-15df-4c84-b580-e6fb1e5e4937 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:36.514973 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 216/416] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:36.520609 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d30199-5c35-4d67-82be-f92d41111691 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:36.521114 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d30199-5c35-4d67-82be-f92d41111691 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:36.521114 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d30199-5c35-4d67-82be-f92d41111691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:36.547660 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-32d30199-5c35-4d67-82be-f92d41111691 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:36.548956 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-32d30199-5c35-4d67-82be-f92d41111691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:36.655902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 201/417] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:17:36.688983 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1b9d7-15df-4c84-b580-e6fb1e5e4937 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:36.829922 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1b9d7-15df-4c84-b580-e6fb1e5e4937 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:36.856347 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:36.889556 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89611eaf-e846-4b75-b0d8-e9e1ee697a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:36.890409 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89611eaf-e846-4b75-b0d8-e9e1ee697a50 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:36.890695 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89611eaf-e846-4b75-b0d8-e9e1ee697a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:36.890976 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89611eaf-e846-4b75-b0d8-e9e1ee697a50 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:36.910785 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89611eaf-e846-4b75-b0d8-e9e1ee697a50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:36.919604 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-89611eaf-e846-4b75-b0d8-e9e1ee697a50 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:17:36.919604 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:36.919604 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:36.919604 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:36.919604 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:36.919604 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:36.919604 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:36.919604 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:36.919604 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:36.919604 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:36.919604 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:36.919604 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:36.919604 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:36.919604 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:36.919604 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:36.919604 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:36.920515 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:36.920515 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:36.920515 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:36.920515 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:36.920515 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:36.920515 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:36.920515 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:36.920515 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:36.920515 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:36.920515 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:36.920515 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:36.920515 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:17:36.920515 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:36.920515 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:36.920515 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:36.920515 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:36.921306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:36.921306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:36.921306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:36.921306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:36.921306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:36.921306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:36.921306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:36.921306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:36.921306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:36.921306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:36.921306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:17:36.921306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:17:36.921306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 217/418] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 21:17:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:36.931417 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d30199-5c35-4d67-82be-f92d41111691 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:37.043554 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:37.087059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d30199-5c35-4d67-82be-f92d41111691 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:37.135245 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43dcb56-f6fa-4463-b315-17f85acde66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:37.135932 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43dcb56-f6fa-4463-b315-17f85acde66b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:37.136209 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43dcb56-f6fa-4463-b315-17f85acde66b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:37.136481 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43dcb56-f6fa-4463-b315-17f85acde66b None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:37.137227 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43dcb56-f6fa-4463-b315-17f85acde66b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:37.137526 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43dcb56-f6fa-4463-b315-17f85acde66b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:37.138924 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43dcb56-f6fa-4463-b315-17f85acde66b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:37.154099 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 202/419] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:17:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:37.159310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89611eaf-e846-4b75-b0d8-e9e1ee697a50 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:37.215733 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcfb7f-0dcc-4896-bb4a-3c04ff2dc6af None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:37.216056 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcfb7f-0dcc-4896-bb4a-3c04ff2dc6af None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:37.216295 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcfb7f-0dcc-4896-bb4a-3c04ff2dc6af None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:37.217940 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 203/420] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:37.222677 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5344c9f-efe2-44db-8222-f5d6d92dd875 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:37.222963 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5344c9f-efe2-44db-8222-f5d6d92dd875 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:37.223242 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5344c9f-efe2-44db-8222-f5d6d92dd875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:37.263387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-d5344c9f-efe2-44db-8222-f5d6d92dd875 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:37.264780 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d5344c9f-efe2-44db-8222-f5d6d92dd875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:37.294071 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89611eaf-e846-4b75-b0d8-e9e1ee697a50 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:37.333455 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9442bb-4d55-49c9-b4d3-c518cf1854ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:37.334120 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9442bb-4d55-49c9-b4d3-c518cf1854ab None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:37.334441 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9442bb-4d55-49c9-b4d3-c518cf1854ab None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:37.334719 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9442bb-4d55-49c9-b4d3-c518cf1854ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:37.381985 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:37.411274 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9442bb-4d55-49c9-b4d3-c518cf1854ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:37.412793 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9442bb-4d55-49c9-b4d3-c518cf1854ab None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:37.436119 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 204/421] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:17:37.474375 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5344c9f-efe2-44db-8222-f5d6d92dd875 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:37.520336 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 218/422] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:17:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:17:37.525512 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9442bb-4d55-49c9-b4d3-c518cf1854ab None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:37.575441 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:37.625321 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5344c9f-efe2-44db-8222-f5d6d92dd875 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:37.669773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12f5590-05e8-4d2f-93b3-616490358e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:37.671249 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f5590-05e8-4d2f-93b3-616490358e1d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:37.671673 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f5590-05e8-4d2f-93b3-616490358e1d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:37.672096 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f5590-05e8-4d2f-93b3-616490358e1d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:37.684846 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12f5590-05e8-4d2f-93b3-616490358e1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:37.693501 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-e12f5590-05e8-4d2f-93b3-616490358e1d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 21:17:37.693501 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:37.693501 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:37.693501 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:37.693501 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:37.693501 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:37.693501 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:37.693501 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:37.693501 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:37.693501 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:37.693501 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:37.693501 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:37.693501 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:37.693501 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:37.693501 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:37.693501 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:37.694563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:37.694563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:37.694563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:37.694563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:37.694563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:37.694563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:37.694563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:37.694563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:37.694563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:37.694563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:37.694563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:37.694563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:37.694563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:37.694563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:37.694563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:37.694563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:37.695461 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:37.695461 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:37.695461 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:37.695461 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:37.695461 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:37.695461 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:37.695461 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:37.695461 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:37.695461 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:37.695461 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:37.695461 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 21:17:37.695461 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:37.696115 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:37.699407 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 205/423] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:17:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:17:37.706941 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12f5590-05e8-4d2f-93b3-616490358e1d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:37.825204 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9442bb-4d55-49c9-b4d3-c518cf1854ab None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:37.830155 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12f5590-05e8-4d2f-93b3-616490358e1d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:37.858475 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:37.866083 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21bf867-b156-4ad5-8566-03d5f063b685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:37.867017 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bf867-b156-4ad5-8566-03d5f063b685 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:37.867786 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bf867-b156-4ad5-8566-03d5f063b685 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:37.868632 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bf867-b156-4ad5-8566-03d5f063b685 None admin] PATH_INFO: `/v3/projects/c12a755e369e4c55a7d4a855fc16d986` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:37.869415 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21bf867-b156-4ad5-8566-03d5f063b685 None admin] RBAC: Authorizing `identity:get_project(project_id=c12a755e369e4c55a7d4a855fc16d986)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:37.871044 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21bf867-b156-4ad5-8566-03d5f063b685 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:37.873197 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 219/424] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:17:37 2022] GET /identity/v3/projects/c12a755e369e4c55a7d4a855fc16d986 => generated 298 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:37.876120 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ddd74-1c9d-49aa-ba19-215bdbbe10b5 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:37.876571 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ddd74-1c9d-49aa-ba19-215bdbbe10b5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:37.876988 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ddd74-1c9d-49aa-ba19-215bdbbe10b5 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:37.878516 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 220/425] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:37.879851 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ddd74-1c9d-49aa-ba19-215bdbbe10b5 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:37.885955 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048f7f84-732d-449d-b54b-df52eaff9569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:37.886684 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f7f84-732d-449d-b54b-df52eaff9569 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:37.886964 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f7f84-732d-449d-b54b-df52eaff9569 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:37.887238 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f7f84-732d-449d-b54b-df52eaff9569 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:37.888799 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048f7f84-732d-449d-b54b-df52eaff9569 None admin] RBAC: Adding query filter params (name=member) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:37.888799 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048f7f84-732d-449d-b54b-df52eaff9569 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:37.889695 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048f7f84-732d-449d-b54b-df52eaff9569 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:37.904116 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 206/426] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 21:17:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:37.904116 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d23e2-2e22-469d-b379-d053f2da9d59 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:37.904116 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d23e2-2e22-469d-b379-d053f2da9d59 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:37.904116 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d23e2-2e22-469d-b379-d053f2da9d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:37.926869 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-ba0d23e2-2e22-469d-b379-d053f2da9d59 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:37.928115 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ba0d23e2-2e22-469d-b379-d053f2da9d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:38.010386 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 207/427] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:17:38.010386 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0d23e2-2e22-469d-b379-d053f2da9d59 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:38.027148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ddd74-1c9d-49aa-ba19-215bdbbe10b5 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:38.073684 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c9e0a-72be-492c-9d9d-5b6db1864d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:38.074825 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c9e0a-72be-492c-9d9d-5b6db1864d7a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:38.075420 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c9e0a-72be-492c-9d9d-5b6db1864d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:38.075835 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c9e0a-72be-492c-9d9d-5b6db1864d7a None admin] PATH_INFO: `/v3/groups/f0c359348291493c8f55343e5f9c5054` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:38.077357 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55c9e0a-72be-492c-9d9d-5b6db1864d7a None admin] RBAC: Authorizing `identity:get_group(group_id=f0c359348291493c8f55343e5f9c5054)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:38.079118 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55c9e0a-72be-492c-9d9d-5b6db1864d7a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:38.081254 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 221/428] 10.222.0.65 () {56 vars in 1180 bytes} [Thu May 19 21:17:37 2022] GET /identity/v3/groups/f0c359348291493c8f55343e5f9c5054 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:38.082381 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c9e0a-72be-492c-9d9d-5b6db1864d7a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:38.094746 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:38.116907 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0d23e2-2e22-469d-b379-d053f2da9d59 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:38.133033 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:38.158405 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f63d17-025d-4194-abe4-12eb423fd401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:38.159408 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f63d17-025d-4194-abe4-12eb423fd401 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:38.159897 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f63d17-025d-4194-abe4-12eb423fd401 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:38.160365 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f63d17-025d-4194-abe4-12eb423fd401 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:38.196624 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c9e0a-72be-492c-9d9d-5b6db1864d7a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:38.233277 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de21edb-272e-487b-abb8-0f53eb7d75dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:38.234207 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de21edb-272e-487b-abb8-0f53eb7d75dc None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:38.234591 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de21edb-272e-487b-abb8-0f53eb7d75dc None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:38.234931 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de21edb-272e-487b-abb8-0f53eb7d75dc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:38.239054 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f63d17-025d-4194-abe4-12eb423fd401 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:38.240735 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f63d17-025d-4194-abe4-12eb423fd401 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:38.246767 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de21edb-272e-487b-abb8-0f53eb7d75dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:38.255148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-3de21edb-272e-487b-abb8-0f53eb7d75dc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 21:17:38.255148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:38.255148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:38.255148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:38.255148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:38.255148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:38.255148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:38.255148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:38.255148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:38.255148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:38.255148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:38.255148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:38.255148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:38.255148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:38.255148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:38.255148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:38.256100 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:38.256100 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:38.256100 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:38.256100 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:38.256100 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:38.256100 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:38.256100 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:38.256100 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:38.256100 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:38.256100 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:38.256100 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:38.256100 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:17:38.256100 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:38.256100 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:38.256100 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:38.256100 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:38.256906 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:38.256906 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:38.256906 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:38.256906 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:38.256906 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:38.256906 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:38.256906 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:38.256906 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:38.256906 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:38.256906 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:38.256906 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 21:17:38.256906 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:17:38.261472 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 222/429] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:17:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:38.262458 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de21edb-272e-487b-abb8-0f53eb7d75dc None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:38.333659 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:38.355683 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 208/430] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:17:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:17:38.356816 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f63d17-025d-4194-abe4-12eb423fd401 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:38.374359 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:38.375811 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de21edb-272e-487b-abb8-0f53eb7d75dc None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:38.418469 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e402792-644f-407c-846a-7ba7434821ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:38.419563 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e402792-644f-407c-846a-7ba7434821ae None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:38.420057 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e402792-644f-407c-846a-7ba7434821ae None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:38.420546 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e402792-644f-407c-846a-7ba7434821ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:38.421706 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e402792-644f-407c-846a-7ba7434821ae None admin] RBAC: Adding query filter params (group.id=f0c359348291493c8f55343e5f9c5054, role.id=566c421659204df185f8b77bf3bb3afc, scope.project.id=c12a755e369e4c55a7d4a855fc16d986) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:38.422233 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e402792-644f-407c-846a-7ba7434821ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:38.423984 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e402792-644f-407c-846a-7ba7434821ae None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:38.431766 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 223/431] 10.222.0.65 () {56 vars in 1376 bytes} [Thu May 19 21:17:38 2022] GET /identity/v3/role_assignments?group.id=f0c359348291493c8f55343e5f9c5054&scope.project.id=c12a755e369e4c55a7d4a855fc16d986&role.id=566c421659204df185f8b77bf3bb3afc => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:38.432981 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e402792-644f-407c-846a-7ba7434821ae None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:38.487126 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f63d17-025d-4194-abe4-12eb423fd401 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:38.525328 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eee6c1-e82c-4198-8045-182c6562e28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:38.526014 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eee6c1-e82c-4198-8045-182c6562e28f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:38.526322 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eee6c1-e82c-4198-8045-182c6562e28f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:38.526608 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eee6c1-e82c-4198-8045-182c6562e28f None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:38.527358 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eee6c1-e82c-4198-8045-182c6562e28f None admin] RBAC: Adding query filter params (name=member) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:38.527664 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eee6c1-e82c-4198-8045-182c6562e28f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:38.529551 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eee6c1-e82c-4198-8045-182c6562e28f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:38.542058 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 209/432] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 21:17:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:38.548884 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eee6c1-e82c-4198-8045-182c6562e28f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:38.567824 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e402792-644f-407c-846a-7ba7434821ae None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:38.653281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:38.663703 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:38.679396 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7024258-22a3-456f-8f80-fec2c110ba05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:38.680359 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7024258-22a3-456f-8f80-fec2c110ba05 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:38.680646 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7024258-22a3-456f-8f80-fec2c110ba05 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:38.680928 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7024258-22a3-456f-8f80-fec2c110ba05 None admin] PATH_INFO: `/v3/projects/c12a755e369e4c55a7d4a855fc16d986` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:38.682400 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7024258-22a3-456f-8f80-fec2c110ba05 None admin] RBAC: Authorizing `identity:get_project(project_id=c12a755e369e4c55a7d4a855fc16d986)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:38.683876 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7024258-22a3-456f-8f80-fec2c110ba05 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:38.686172 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 224/433] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:17:38 2022] GET /identity/v3/projects/c12a755e369e4c55a7d4a855fc16d986 => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:38.693710 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550cf85-3762-466d-93cc-dc9f55fdbf07 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:38.693710 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550cf85-3762-466d-93cc-dc9f55fdbf07 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:38.693710 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550cf85-3762-466d-93cc-dc9f55fdbf07 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:38.693710 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 225/434] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:38.693710 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8550cf85-3762-466d-93cc-dc9f55fdbf07 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:38.705739 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eee6c1-e82c-4198-8045-182c6562e28f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:38.755359 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eda175-a64d-4ba3-aa41-4eb9b6b867d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:38.756375 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eda175-a64d-4ba3-aa41-4eb9b6b867d1 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:38.756833 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eda175-a64d-4ba3-aa41-4eb9b6b867d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:38.757425 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eda175-a64d-4ba3-aa41-4eb9b6b867d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:38.825354 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8550cf85-3762-466d-93cc-dc9f55fdbf07 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:38.832575 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eda175-a64d-4ba3-aa41-4eb9b6b867d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:38.834179 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eda175-a64d-4ba3-aa41-4eb9b6b867d1 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:38.871373 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf79a9-7d3e-42ce-9348-c7af0c317389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:38.872297 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf79a9-7d3e-42ce-9348-c7af0c317389 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:38.872578 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf79a9-7d3e-42ce-9348-c7af0c317389 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:38.872791 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf79a9-7d3e-42ce-9348-c7af0c317389 None admin] PATH_INFO: `/v3/groups/f0c359348291493c8f55343e5f9c5054` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:38.874028 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf79a9-7d3e-42ce-9348-c7af0c317389 None admin] RBAC: Authorizing `identity:get_group(group_id=f0c359348291493c8f55343e5f9c5054)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:38.875496 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf79a9-7d3e-42ce-9348-c7af0c317389 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:38.877174 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 226/435] 10.222.0.65 () {56 vars in 1180 bytes} [Thu May 19 21:17:38 2022] GET /identity/v3/groups/f0c359348291493c8f55343e5f9c5054 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:38.883186 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8334461-1aed-4c8a-bc37-b536fa136183 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:38.883466 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8334461-1aed-4c8a-bc37-b536fa136183 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:38.883743 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8334461-1aed-4c8a-bc37-b536fa136183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:38.901572 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b8334461-1aed-4c8a-bc37-b536fa136183 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:38.903026 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b8334461-1aed-4c8a-bc37-b536fa136183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:38.933268 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:38.961123 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 210/436] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:17:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 21:17:38.962291 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eda175-a64d-4ba3-aa41-4eb9b6b867d1 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:38.977301 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:38.986286 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 227/437] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:17:38.986286 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8334461-1aed-4c8a-bc37-b536fa136183 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:39.096782 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eda175-a64d-4ba3-aa41-4eb9b6b867d1 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:39.100332 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8334461-1aed-4c8a-bc37-b536fa136183 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:39.133453 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685fcb20-c879-4279-aeb7-c1f274395009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:39.134016 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685fcb20-c879-4279-aeb7-c1f274395009 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:39.134328 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685fcb20-c879-4279-aeb7-c1f274395009 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:39.134606 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685fcb20-c879-4279-aeb7-c1f274395009 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:39.135438 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685fcb20-c879-4279-aeb7-c1f274395009 None admin] RBAC: Adding query filter params (group.id=f0c359348291493c8f55343e5f9c5054, role.id=70699fb0eae7441999835fa3b50350b3, scope.project.id=c12a755e369e4c55a7d4a855fc16d986) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:39.135747 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685fcb20-c879-4279-aeb7-c1f274395009 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:39.136459 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c65d7-ceb5-47c9-a8de-d92cba5f21bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:39.137211 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685fcb20-c879-4279-aeb7-c1f274395009 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:39.137331 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c65d7-ceb5-47c9-a8de-d92cba5f21bb None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:39.137616 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c65d7-ceb5-47c9-a8de-d92cba5f21bb None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:39.137891 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c65d7-ceb5-47c9-a8de-d92cba5f21bb None admin] PATH_INFO: `/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:39.139135 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168c65d7-ceb5-47c9-a8de-d92cba5f21bb None admin] RBAC: Authorizing `identity:get_project(project_id=1264ceb6aa114fbf81b3d48cd8874c8f)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:39.140640 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168c65d7-ceb5-47c9-a8de-d92cba5f21bb None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:39.146890 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 228/438] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:17:38 2022] GET /identity/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:39.147061 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 211/439] 10.222.0.65 () {56 vars in 1376 bytes} [Thu May 19 21:17:38 2022] GET /identity/v3/role_assignments?group.id=f0c359348291493c8f55343e5f9c5054&scope.project.id=c12a755e369e4c55a7d4a855fc16d986&role.id=70699fb0eae7441999835fa3b50350b3 => generated 611 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:39.147172 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c65d7-ceb5-47c9-a8de-d92cba5f21bb None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:39.147245 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bd5e5-8396-4f01-bf85-6ce912e8d8b7 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:39.147245 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bd5e5-8396-4f01-bf85-6ce912e8d8b7 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:39.147245 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bd5e5-8396-4f01-bf85-6ce912e8d8b7 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:39.147245 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 212/440] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:39.152037 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315bd5e5-8396-4f01-bf85-6ce912e8d8b7 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:39.205712 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:39.234142 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:39.280579 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315bd5e5-8396-4f01-bf85-6ce912e8d8b7 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:39.285169 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c65d7-ceb5-47c9-a8de-d92cba5f21bb None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:39.321973 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec058d-ccda-45f4-a44a-12c84a00de73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:39.322893 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec058d-ccda-45f4-a44a-12c84a00de73 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:39.323182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec058d-ccda-45f4-a44a-12c84a00de73 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:39.323439 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec058d-ccda-45f4-a44a-12c84a00de73 None admin] PATH_INFO: `/v3/groups/f0c359348291493c8f55343e5f9c5054` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:39.324681 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ec058d-ccda-45f4-a44a-12c84a00de73 None admin] RBAC: Authorizing `identity:get_group(group_id=f0c359348291493c8f55343e5f9c5054)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:39.325292 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a70d28-623d-416a-959e-76e81475a479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:39.326185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a70d28-623d-416a-959e-76e81475a479 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:39.326316 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ec058d-ccda-45f4-a44a-12c84a00de73 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:39.326602 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a70d28-623d-416a-959e-76e81475a479 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:39.326936 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a70d28-623d-416a-959e-76e81475a479 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:39.328069 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 213/441] 10.222.0.65 () {56 vars in 1180 bytes} [Thu May 19 21:17:39 2022] GET /identity/v3/groups/f0c359348291493c8f55343e5f9c5054 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:39.329281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4520e-351d-4eea-8324-ccaf38e98312 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:39.329563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4520e-351d-4eea-8324-ccaf38e98312 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:39.329832 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4520e-351d-4eea-8324-ccaf38e98312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:39.348161 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a70d28-623d-416a-959e-76e81475a479 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:39.350514 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-45d4520e-351d-4eea-8324-ccaf38e98312 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:39.352551 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-45d4520e-351d-4eea-8324-ccaf38e98312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:39.356758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-94a70d28-623d-416a-959e-76e81475a479 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:17:39.356758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:39.356758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:39.356758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:39.356758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:39.356758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:39.356758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:39.356758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:39.356758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:39.356758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:39.356758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:39.356758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:39.356758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:39.356758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:39.356758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:39.356758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:39.357846 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:39.357846 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:39.357846 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:39.357846 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:39.357846 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:39.357846 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:39.357846 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:39.357846 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:39.357846 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:39.357846 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:39.357846 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:39.357846 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:17:39.357846 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:39.357846 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:39.357846 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:39.357846 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:39.358735 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:39.358735 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:39.358735 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:39.358735 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:39.358735 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:39.358735 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:39.358735 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:39.358735 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:39.358735 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:39.358735 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:39.358735 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:17:39.358735 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:17:39.358735 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 229/442] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 21:17:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:39.359379 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a70d28-623d-416a-959e-76e81475a479 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:39.438264 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 214/443] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:17:39.438264 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d4520e-351d-4eea-8324-ccaf38e98312 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:39.480458 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:39.490340 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a70d28-623d-416a-959e-76e81475a479 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:39.524151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:39.539119 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c645327-901a-45f7-87c1-5bead0128a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:39.539794 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c645327-901a-45f7-87c1-5bead0128a94 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:39.540080 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c645327-901a-45f7-87c1-5bead0128a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:39.540357 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c645327-901a-45f7-87c1-5bead0128a94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:39.541198 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c645327-901a-45f7-87c1-5bead0128a94 None admin] RBAC: Adding query filter params (group.id=f0c359348291493c8f55343e5f9c5054, role.id=70699fb0eae7441999835fa3b50350b3, scope.project.id=1264ceb6aa114fbf81b3d48cd8874c8f) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:39.541509 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c645327-901a-45f7-87c1-5bead0128a94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:39.543015 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c645327-901a-45f7-87c1-5bead0128a94 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:39.549655 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 230/444] 10.222.0.65 () {56 vars in 1376 bytes} [Thu May 19 21:17:39 2022] GET /identity/v3/role_assignments?group.id=f0c359348291493c8f55343e5f9c5054&scope.project.id=1264ceb6aa114fbf81b3d48cd8874c8f&role.id=70699fb0eae7441999835fa3b50350b3 => generated 611 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:39.550639 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c645327-901a-45f7-87c1-5bead0128a94 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:39.583470 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d4520e-351d-4eea-8324-ccaf38e98312 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:39.641091 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf8698a-6f92-4f99-8bf7-e7a5ce313d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:39.641900 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8698a-6f92-4f99-8bf7-e7a5ce313d2a None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:39.642194 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8698a-6f92-4f99-8bf7-e7a5ce313d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:39.642505 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8698a-6f92-4f99-8bf7-e7a5ce313d2a None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:39.643937 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf8698a-6f92-4f99-8bf7-e7a5ce313d2a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:39.644254 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf8698a-6f92-4f99-8bf7-e7a5ce313d2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:39.645687 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf8698a-6f92-4f99-8bf7-e7a5ce313d2a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:39.658309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 215/445] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:17:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:39.662990 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf8698a-6f92-4f99-8bf7-e7a5ce313d2a None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:39.688873 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c645327-901a-45f7-87c1-5bead0128a94 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:39.741827 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c592330-40dd-41d5-9e27-b87a9a71a2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:39.743487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c592330-40dd-41d5-9e27-b87a9a71a2e9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:39.743734 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c592330-40dd-41d5-9e27-b87a9a71a2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:39.744010 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c592330-40dd-41d5-9e27-b87a9a71a2e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:39.761598 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c592330-40dd-41d5-9e27-b87a9a71a2e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:39.769446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-0c592330-40dd-41d5-9e27-b87a9a71a2e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:17:39.769446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:39.769446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:39.769446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:39.769446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:39.769446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:39.769446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:39.769446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:39.769446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:39.769446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:39.769446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:39.769446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:39.769446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:39.769446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:39.769446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:39.769446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:39.770671 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:39.770671 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:39.770671 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:39.770671 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:39.770671 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:39.770671 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:39.770671 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:39.770671 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:39.770671 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:39.770671 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:39.770671 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:39.770671 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:17:39.770671 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:39.770671 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:39.770671 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:39.770671 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:39.771485 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:39.771485 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:39.771485 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:39.771485 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:39.771485 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:39.771485 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:39.771485 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:39.771485 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:39.771485 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:39.771485 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:39.771485 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:17:39.771485 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:17:39.771485 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 231/446] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 21:17:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:39.777845 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c592330-40dd-41d5-9e27-b87a9a71a2e9 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:39.811620 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:39.818561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf8698a-6f92-4f99-8bf7-e7a5ce313d2a None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:39.847131 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:39.868575 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adaece4-4b20-4e64-9517-52707e69b8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:39.869738 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adaece4-4b20-4e64-9517-52707e69b8e2 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:39.869738 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adaece4-4b20-4e64-9517-52707e69b8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:39.870606 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adaece4-4b20-4e64-9517-52707e69b8e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:39.921699 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c592330-40dd-41d5-9e27-b87a9a71a2e9 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:39.961673 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f0ab8-121d-4d6a-9896-d4ad6f36632c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:39.961904 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f0ab8-121d-4d6a-9896-d4ad6f36632c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:39.962193 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f0ab8-121d-4d6a-9896-d4ad6f36632c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:39.962501 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f0ab8-121d-4d6a-9896-d4ad6f36632c None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:39.963256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f0ab8-121d-4d6a-9896-d4ad6f36632c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:39.963561 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f0ab8-121d-4d6a-9896-d4ad6f36632c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:39.964002 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adaece4-4b20-4e64-9517-52707e69b8e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:39.964945 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f0ab8-121d-4d6a-9896-d4ad6f36632c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:39.965465 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adaece4-4b20-4e64-9517-52707e69b8e2 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:39.978072 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 232/447] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:17:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:39.983447 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f0ab8-121d-4d6a-9896-d4ad6f36632c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:40.071433 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:40.078414 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 216/448] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:17:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 21:17:40.082352 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adaece4-4b20-4e64-9517-52707e69b8e2 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:40.103376 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f0ab8-121d-4d6a-9896-d4ad6f36632c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:40.118520 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:40.143409 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae734bc-2f4e-4590-8b22-55c94546d711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:40.144062 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae734bc-2f4e-4590-8b22-55c94546d711 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:40.144343 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae734bc-2f4e-4590-8b22-55c94546d711 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:40.144615 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae734bc-2f4e-4590-8b22-55c94546d711 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:40.199583 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adaece4-4b20-4e64-9517-52707e69b8e2 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:40.220016 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae734bc-2f4e-4590-8b22-55c94546d711 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:40.221516 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae734bc-2f4e-4590-8b22-55c94546d711 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:40.245574 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0599751-c1f7-45ae-ad10-18a63f62d17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:40.246507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0599751-c1f7-45ae-ad10-18a63f62d17c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:40.246936 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0599751-c1f7-45ae-ad10-18a63f62d17c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:40.247409 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0599751-c1f7-45ae-ad10-18a63f62d17c None admin] PATH_INFO: `/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:40.249285 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0599751-c1f7-45ae-ad10-18a63f62d17c None admin] RBAC: Authorizing `identity:get_project(project_id=1264ceb6aa114fbf81b3d48cd8874c8f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:40.250799 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0599751-c1f7-45ae-ad10-18a63f62d17c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:40.252518 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 217/449] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:17:40 2022] GET /identity/v3/projects/1264ceb6aa114fbf81b3d48cd8874c8f => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:17:40.258140 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0599751-c1f7-45ae-ad10-18a63f62d17c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:40.323540 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:40.334593 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 233/450] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:17:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 21:17:40.340530 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae734bc-2f4e-4590-8b22-55c94546d711 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:40.354663 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:40.360523 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0599751-c1f7-45ae-ad10-18a63f62d17c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:40.396096 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e242908b-826e-4db3-9f59-6284661d690d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:40.396959 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e242908b-826e-4db3-9f59-6284661d690d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:40.397241 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e242908b-826e-4db3-9f59-6284661d690d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:40.397523 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e242908b-826e-4db3-9f59-6284661d690d None admin] PATH_INFO: `/v3/groups/f0c359348291493c8f55343e5f9c5054` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:40.398827 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e242908b-826e-4db3-9f59-6284661d690d None admin] RBAC: Authorizing `identity:get_group(group_id=f0c359348291493c8f55343e5f9c5054)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:40.400328 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e242908b-826e-4db3-9f59-6284661d690d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:40.401974 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 218/451] 10.222.0.65 () {56 vars in 1180 bytes} [Thu May 19 21:17:40 2022] GET /identity/v3/groups/f0c359348291493c8f55343e5f9c5054 => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:17:40.407752 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e242908b-826e-4db3-9f59-6284661d690d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:40.443521 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae734bc-2f4e-4590-8b22-55c94546d711 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:40.478770 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a93994-bbb4-4516-b141-98692791cd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:40.479557 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a93994-bbb4-4516-b141-98692791cd93 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:40.479838 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a93994-bbb4-4516-b141-98692791cd93 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:40.480112 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a93994-bbb4-4516-b141-98692791cd93 None admin] PATH_INFO: `/v3/projects/0436212ebbfa4ccb91d5457faaa851ee` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:40.481308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a93994-bbb4-4516-b141-98692791cd93 None admin] RBAC: Authorizing `identity:get_project(project_id=0436212ebbfa4ccb91d5457faaa851ee)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:40.482786 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a93994-bbb4-4516-b141-98692791cd93 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:40.484483 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 234/452] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:17:40 2022] GET /identity/v3/projects/0436212ebbfa4ccb91d5457faaa851ee => generated 344 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:17:40.489917 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a93994-bbb4-4516-b141-98692791cd93 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:40.507366 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e242908b-826e-4db3-9f59-6284661d690d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:40.542379 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae76a9-aa43-4200-bdd0-8861c0d421f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:40.543047 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae76a9-aa43-4200-bdd0-8861c0d421f5 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:40.543325 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae76a9-aa43-4200-bdd0-8861c0d421f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:40.543600 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae76a9-aa43-4200-bdd0-8861c0d421f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:40.544437 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae76a9-aa43-4200-bdd0-8861c0d421f5 None admin] RBAC: Adding query filter params (group.id=f0c359348291493c8f55343e5f9c5054, role.id=566c421659204df185f8b77bf3bb3afc, scope.project.id=1264ceb6aa114fbf81b3d48cd8874c8f) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:40.544834 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae76a9-aa43-4200-bdd0-8861c0d421f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:40.546196 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae76a9-aa43-4200-bdd0-8861c0d421f5 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:40.548203 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:40.553799 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 219/453] 10.222.0.65 () {56 vars in 1376 bytes} [Thu May 19 21:17:40 2022] GET /identity/v3/role_assignments?group.id=f0c359348291493c8f55343e5f9c5054&scope.project.id=1264ceb6aa114fbf81b3d48cd8874c8f&role.id=566c421659204df185f8b77bf3bb3afc => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:17:40.575877 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:40.598740 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a93994-bbb4-4516-b141-98692791cd93 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:40.640795 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83caf8e5-968f-47c1-9ff6-bed620c0bb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:40.641688 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83caf8e5-968f-47c1-9ff6-bed620c0bb0d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:40.641970 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83caf8e5-968f-47c1-9ff6-bed620c0bb0d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:40.642271 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83caf8e5-968f-47c1-9ff6-bed620c0bb0d None admin] PATH_INFO: `/v3/groups/d4def6f7c4e547e9aa539ed5b7ba04ce` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:40.643674 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83caf8e5-968f-47c1-9ff6-bed620c0bb0d None admin] RBAC: Authorizing `identity:get_group(group_id=d4def6f7c4e547e9aa539ed5b7ba04ce)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:40.645172 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83caf8e5-968f-47c1-9ff6-bed620c0bb0d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:40.646887 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 235/454] 10.222.0.65 () {56 vars in 1180 bytes} [Thu May 19 21:17:40 2022] GET /identity/v3/groups/d4def6f7c4e547e9aa539ed5b7ba04ce => generated 229 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:40.653320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae76a9-aa43-4200-bdd0-8861c0d421f5 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:40.766205 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae76a9-aa43-4200-bdd0-8861c0d421f5 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:40.800567 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28502bcd-f1e2-420d-a242-12fefd2e50da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:40.801233 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28502bcd-f1e2-420d-a242-12fefd2e50da None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:40.801516 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28502bcd-f1e2-420d-a242-12fefd2e50da None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:40.801791 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28502bcd-f1e2-420d-a242-12fefd2e50da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:40.802663 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28502bcd-f1e2-420d-a242-12fefd2e50da None admin] RBAC: Adding query filter params (group.id=d4def6f7c4e547e9aa539ed5b7ba04ce, role.id=2b714f2e5c6c469f8fa037bda1ed16f6, scope.project.id=0436212ebbfa4ccb91d5457faaa851ee) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:40.802979 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28502bcd-f1e2-420d-a242-12fefd2e50da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:40.804468 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28502bcd-f1e2-420d-a242-12fefd2e50da None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:40.812038 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 220/455] 10.222.0.65 () {56 vars in 1376 bytes} [Thu May 19 21:17:40 2022] GET /identity/v3/role_assignments?group.id=d4def6f7c4e547e9aa539ed5b7ba04ce&scope.project.id=0436212ebbfa4ccb91d5457faaa851ee&role.id=2b714f2e5c6c469f8fa037bda1ed16f6 => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:43.599576 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285b4a5-0762-4d8e-99b2-3d85a16e2b2c None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:43.600144 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285b4a5-0762-4d8e-99b2-3d85a16e2b2c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:43.600144 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285b4a5-0762-4d8e-99b2-3d85a16e2b2c None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:43.602059 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 236/456] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:43.607702 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6d4e1-3b19-4948-8e3c-6408040cbabf None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:43.608562 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6d4e1-3b19-4948-8e3c-6408040cbabf None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:43.609014 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6d4e1-3b19-4948-8e3c-6408040cbabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:43.631494 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-10c6d4e1-3b19-4948-8e3c-6408040cbabf None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:43.633134 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-10c6d4e1-3b19-4948-8e3c-6408040cbabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:43.728600 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:43.753180 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 221/457] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:17:43.790012 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285b4a5-0762-4d8e-99b2-3d85a16e2b2c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:43.845836 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f168e-e206-4d29-be3f-c9ef956e8fe2 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:43.845836 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f168e-e206-4d29-be3f-c9ef956e8fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:43.846082 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f168e-e206-4d29-be3f-c9ef956e8fe2 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:43.847957 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 222/458] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:43.853485 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52925ce2-6fd8-474d-b80b-4d8a01b92edc None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:43.853986 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52925ce2-6fd8-474d-b80b-4d8a01b92edc None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:43.854455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52925ce2-6fd8-474d-b80b-4d8a01b92edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:43.875383 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-52925ce2-6fd8-474d-b80b-4d8a01b92edc None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:43.876666 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-52925ce2-6fd8-474d-b80b-4d8a01b92edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:43.934604 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285b4a5-0762-4d8e-99b2-3d85a16e2b2c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:43.960323 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 223/459] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:17:43.961804 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19c086-bcc7-4e05-80c3-d9ff2946db09 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:43.962095 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19c086-bcc7-4e05-80c3-d9ff2946db09 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:43.962413 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19c086-bcc7-4e05-80c3-d9ff2946db09 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:43.966381 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 224/460] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:43.971553 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec60d3a-c3b0-450b-975c-37a81e7c2e9c None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:43.971846 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec60d3a-c3b0-450b-975c-37a81e7c2e9c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:43.972130 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec60d3a-c3b0-450b-975c-37a81e7c2e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:43.982668 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5795fd-045a-4bc2-8ade-83f5b896fb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:43.983812 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5795fd-045a-4bc2-8ade-83f5b896fb31 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:43.984284 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5795fd-045a-4bc2-8ade-83f5b896fb31 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:43.984742 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5795fd-045a-4bc2-8ade-83f5b896fb31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:43.992914 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-bec60d3a-c3b0-450b-975c-37a81e7c2e9c None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:43.994737 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-bec60d3a-c3b0-450b-975c-37a81e7c2e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:43.998380 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5795fd-045a-4bc2-8ade-83f5b896fb31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:44.000884 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5795fd-045a-4bc2-8ade-83f5b896fb31 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:44.012528 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:44.025265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-df5795fd-045a-4bc2-8ade-83f5b896fb31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:44.025265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:44.025265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:44.025265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:44.025265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:44.025265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:44.025265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:44.025265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:44.025265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:44.025265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:44.025265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:44.025265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:44.025265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:17:44.025265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:17:44.025265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:17:44.026161 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:17:44.026161 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:44.026161 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:44.026161 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:44.026161 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:44.026161 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:44.026161 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:44.026161 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:44.026161 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:17:44.026161 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:44.026161 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:44.026161 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:44.026161 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:44.026161 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:44.026161 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:44.026161 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:44.026161 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:44.027032 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:17:44.027032 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:17:44.027032 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:44.027032 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:17:44.032596 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 237/461] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:17:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:44.032596 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5795fd-045a-4bc2-8ade-83f5b896fb31 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:44.078719 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 225/462] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:17:44.079921 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec60d3a-c3b0-450b-975c-37a81e7c2e9c None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:44.133771 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:44.184252 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5795fd-045a-4bc2-8ade-83f5b896fb31 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:44.195373 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec60d3a-c3b0-450b-975c-37a81e7c2e9c None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:44.223518 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d8bb71-71d8-4da4-87ba-5d24cef8f3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:44.224672 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8bb71-71d8-4da4-87ba-5d24cef8f3e6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:44.225103 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8bb71-71d8-4da4-87ba-5d24cef8f3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:44.225521 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8bb71-71d8-4da4-87ba-5d24cef8f3e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:44.232568 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1dc6b-6db5-4172-a0d4-ef1aa66c3f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:44.233455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1dc6b-6db5-4172-a0d4-ef1aa66c3f69 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:44.233870 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1dc6b-6db5-4172-a0d4-ef1aa66c3f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:44.234308 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1dc6b-6db5-4172-a0d4-ef1aa66c3f69 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:44.235235 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1dc6b-6db5-4172-a0d4-ef1aa66c3f69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:44.235767 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1dc6b-6db5-4172-a0d4-ef1aa66c3f69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:44.237345 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1dc6b-6db5-4172-a0d4-ef1aa66c3f69 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:44.238308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d8bb71-71d8-4da4-87ba-5d24cef8f3e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:44.239913 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d8bb71-71d8-4da4-87ba-5d24cef8f3e6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:44.251432 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 226/463] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:44.252768 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1dc6b-6db5-4172-a0d4-ef1aa66c3f69 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:44.267251 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-e3d8bb71-71d8-4da4-87ba-5d24cef8f3e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:44.267251 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:44.267251 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:44.267251 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:44.267251 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:44.267251 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:44.267251 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:44.267251 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:44.267251 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:44.267251 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:44.267251 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:44.267251 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:44.267251 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:17:44.267251 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:17:44.267251 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:17:44.268205 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:17:44.268205 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:44.268205 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:44.268205 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:44.268205 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:44.268205 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:44.268205 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:44.268205 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:44.268205 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:17:44.268205 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:44.268205 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:44.268205 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:44.268205 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:44.268205 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:44.268205 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:44.268205 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:44.268205 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:44.269830 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:17:44.269830 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:17:44.269830 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:44.269830 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:17:44.269830 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 238/464] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:17:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:44.269830 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d8bb71-71d8-4da4-87ba-5d24cef8f3e6 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:44.329505 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:44.366729 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1dc6b-6db5-4172-a0d4-ef1aa66c3f69 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:44.381696 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:44.385571 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d8bb71-71d8-4da4-87ba-5d24cef8f3e6 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:44.404883 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1dcb11-cf59-4f38-a02a-38a3526cbeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:44.405689 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1dcb11-cf59-4f38-a02a-38a3526cbeb4 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:44.405966 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1dcb11-cf59-4f38-a02a-38a3526cbeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:44.406260 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1dcb11-cf59-4f38-a02a-38a3526cbeb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:44.418038 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1dcb11-cf59-4f38-a02a-38a3526cbeb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:44.420489 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1dcb11-cf59-4f38-a02a-38a3526cbeb4 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:44.424291 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f24e8-8ac7-4fc8-b268-1ff765682e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:44.424979 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f24e8-8ac7-4fc8-b268-1ff765682e17 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:44.425259 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f24e8-8ac7-4fc8-b268-1ff765682e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:44.425532 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f24e8-8ac7-4fc8-b268-1ff765682e17 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:44.426271 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f24e8-8ac7-4fc8-b268-1ff765682e17 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:44.427741 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f24e8-8ac7-4fc8-b268-1ff765682e17 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:44.440502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-9f1dcb11-cf59-4f38-a02a-38a3526cbeb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:44.440502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:44.440502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:44.440502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:44.440502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:44.440502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:44.440502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:44.440502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:44.440502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:44.440502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:44.440502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:44.440502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:44.440502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:17:44.440502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:17:44.440502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:17:44.441563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:17:44.441563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:44.441563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:44.441563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:44.441563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:44.441563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:44.441563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:44.441563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:44.441563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:44.441563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:44.441563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:44.441563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:44.441563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:44.441563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:44.441563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:44.441563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:44.441563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:44.442440 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:17:44.442440 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:17:44.442440 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:44.442440 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:44.442440 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 227/465] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:17:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:44.442727 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1dcb11-cf59-4f38-a02a-38a3526cbeb4 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:44.462561 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 239/466] 10.222.0.65 () {60 vars in 1132 bytes} [Thu May 19 21:17:44 2022] POST /identity/v3/users => generated 246 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:17:44.462561 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e04f9a-eb4a-4721-8927-0bb7f54f8733 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:44.462561 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e04f9a-eb4a-4721-8927-0bb7f54f8733 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:44.462561 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e04f9a-eb4a-4721-8927-0bb7f54f8733 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:44.462561 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 240/467] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:44.462561 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e04f9a-eb4a-4721-8927-0bb7f54f8733 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:44.572498 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1dcb11-cf59-4f38-a02a-38a3526cbeb4 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:44.602918 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e04f9a-eb4a-4721-8927-0bb7f54f8733 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:44.612299 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d1c0d7-b07e-4d16-921b-95c7724f7ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:44.613077 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d1c0d7-b07e-4d16-921b-95c7724f7ac2 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:44.613245 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d1c0d7-b07e-4d16-921b-95c7724f7ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:44.613522 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d1c0d7-b07e-4d16-921b-95c7724f7ac2 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:44.614269 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d1c0d7-b07e-4d16-921b-95c7724f7ac2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:44.614604 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d1c0d7-b07e-4d16-921b-95c7724f7ac2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:44.616585 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d1c0d7-b07e-4d16-921b-95c7724f7ac2 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:44.616700 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:44.631645 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 228/468] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:44.637967 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac256bea-f505-4dcb-af43-f10cf9a3ea0b None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:44.638154 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac256bea-f505-4dcb-af43-f10cf9a3ea0b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:44.638462 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac256bea-f505-4dcb-af43-f10cf9a3ea0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:44.654875 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c443b-2f30-4071-8b10-ba65ccb29e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:44.655704 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c443b-2f30-4071-8b10-ba65ccb29e63 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:44.655986 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c443b-2f30-4071-8b10-ba65ccb29e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:44.656159 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-ac256bea-f505-4dcb-af43-f10cf9a3ea0b None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:44.656256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c443b-2f30-4071-8b10-ba65ccb29e63 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:44.656992 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c443b-2f30-4071-8b10-ba65ccb29e63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:44.657294 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c443b-2f30-4071-8b10-ba65ccb29e63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:44.657399 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ac256bea-f505-4dcb-af43-f10cf9a3ea0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:44.658767 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c443b-2f30-4071-8b10-ba65ccb29e63 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:44.679911 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 241/469] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:44.681047 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c443b-2f30-4071-8b10-ba65ccb29e63 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:44.685839 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:44.757813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 229/470] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:17:44.760531 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac256bea-f505-4dcb-af43-f10cf9a3ea0b None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:44.837206 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c443b-2f30-4071-8b10-ba65ccb29e63 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:44.887122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1062bc34-36ca-4e11-8462-5679d45eafa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:44.888250 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062bc34-36ca-4e11-8462-5679d45eafa6 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:44.888833 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062bc34-36ca-4e11-8462-5679d45eafa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:44.889312 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062bc34-36ca-4e11-8462-5679d45eafa6 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:44.890311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1062bc34-36ca-4e11-8462-5679d45eafa6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:44.892009 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1062bc34-36ca-4e11-8462-5679d45eafa6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:44.892403 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac256bea-f505-4dcb-af43-f10cf9a3ea0b None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:44.918943 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 242/471] 10.222.0.65 () {60 vars in 1132 bytes} [Thu May 19 21:17:44 2022] POST /identity/v3/users => generated 248 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:17:44.920250 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1062bc34-36ca-4e11-8462-5679d45eafa6 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:44.948381 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da4d21-7a07-4f9a-9294-bcea23e6be1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:44.949268 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da4d21-7a07-4f9a-9294-bcea23e6be1d None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:44.949577 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da4d21-7a07-4f9a-9294-bcea23e6be1d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:44.949946 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da4d21-7a07-4f9a-9294-bcea23e6be1d None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:44.951094 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8da4d21-7a07-4f9a-9294-bcea23e6be1d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:44.955968 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8da4d21-7a07-4f9a-9294-bcea23e6be1d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:44.959045 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:44.988091 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 230/472] 10.222.0.65 () {60 vars in 1132 bytes} [Thu May 19 21:17:44 2022] POST /identity/v3/users => generated 248 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:17:44.995678 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeea220-5b74-434e-b24b-5ef0ac991868 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:44.995678 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeea220-5b74-434e-b24b-5ef0ac991868 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:44.996078 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeea220-5b74-434e-b24b-5ef0ac991868 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:44.998067 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 231/473] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:44 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:45.005049 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb39d8b-74f7-4e2c-95f3-0f23c03ec9f0 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:45.005500 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb39d8b-74f7-4e2c-95f3-0f23c03ec9f0 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:45.005602 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb39d8b-74f7-4e2c-95f3-0f23c03ec9f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:45.038430 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-5eb39d8b-74f7-4e2c-95f3-0f23c03ec9f0 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:45.039727 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5eb39d8b-74f7-4e2c-95f3-0f23c03ec9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:45.043369 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:45.064098 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1062bc34-36ca-4e11-8462-5679d45eafa6 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:45.121805 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7b052-c402-47ac-aaa6-d6af0f4c7013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:45.124591 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7b052-c402-47ac-aaa6-d6af0f4c7013 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:45.125626 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7b052-c402-47ac-aaa6-d6af0f4c7013 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:45.126483 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7b052-c402-47ac-aaa6-d6af0f4c7013 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:45.152263 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa7b052-c402-47ac-aaa6-d6af0f4c7013 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:45.156414 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa7b052-c402-47ac-aaa6-d6af0f4c7013 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:45.188623 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-5aa7b052-c402-47ac-aaa6-d6af0f4c7013 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:45.188623 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:45.188623 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:45.188623 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:45.188623 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:45.188623 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:45.188623 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:45.188623 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:45.188623 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:45.188623 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:45.188623 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:45.188623 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:45.188623 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:17:45.188623 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:17:45.188623 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:17:45.189578 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:17:45.189578 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:45.189578 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:45.189578 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:45.189578 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:45.189578 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:45.189578 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:45.189578 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:45.189578 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:17:45.189578 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:45.189578 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:45.189578 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:45.189578 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:45.189578 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:45.189578 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:45.189578 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:45.189578 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:45.190638 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:17:45.190638 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:17:45.190638 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:45.190638 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:17:45.198077 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 243/474] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:17:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:45.199329 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7b052-c402-47ac-aaa6-d6af0f4c7013 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:45.208320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 232/475] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:17:45.232416 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb39d8b-74f7-4e2c-95f3-0f23c03ec9f0 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:45.312296 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:45.359356 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7b052-c402-47ac-aaa6-d6af0f4c7013 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:45.388485 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb39d8b-74f7-4e2c-95f3-0f23c03ec9f0 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:45.408159 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a4044-cae2-48cc-b24f-82192bd10eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:45.408946 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a4044-cae2-48cc-b24f-82192bd10eee None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:45.409310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a4044-cae2-48cc-b24f-82192bd10eee None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:45.409691 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a4044-cae2-48cc-b24f-82192bd10eee None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:45.410652 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6a4044-cae2-48cc-b24f-82192bd10eee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:45.410973 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6a4044-cae2-48cc-b24f-82192bd10eee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:45.412475 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6a4044-cae2-48cc-b24f-82192bd10eee None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:45.426608 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 244/476] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:17:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:45.435454 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a4044-cae2-48cc-b24f-82192bd10eee None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:45.441589 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ba637-4eb0-45e3-9508-f968589536e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:45.442323 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ba637-4eb0-45e3-9508-f968589536e7 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:45.442608 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ba637-4eb0-45e3-9508-f968589536e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:45.442883 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ba637-4eb0-45e3-9508-f968589536e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:45.443656 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1ba637-4eb0-45e3-9508-f968589536e7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:45.445113 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1ba637-4eb0-45e3-9508-f968589536e7 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:45.456192 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.sql.core [None req-ca1ba637-4eb0-45e3-9508-f968589536e7 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 21:17:45.456192 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 21:17:45.456192 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [parameters: {'id': 'cc0e49ceac1e4dbfa80350d0e6ede415', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 21:17:45.456192 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80270) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 21:17:45.490535 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-ca1ba637-4eb0-45e3-9508-f968589536e7 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 21:17:45.490535 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:45.490535 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 21:17:45.490535 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 21:17:45.490535 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 21:17:45.490535 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 21:17:45.490535 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 21:17:45.490535 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application result = self._query(query) May 19 21:17:45.490535 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 21:17:45.490535 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application conn.query(q) May 19 21:17:45.490535 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 21:17:45.490535 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 21:17:45.490535 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 21:17:45.490535 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application result.read() May 19 21:17:45.490535 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 21:17:45.491522 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 21:17:45.491522 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 21:17:45.491522 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 21:17:45.491522 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 21:17:45.491522 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 21:17:45.491522 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 21:17:45.491522 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 21:17:45.491522 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 21:17:45.491522 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:45.491522 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 21:17:45.491522 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:45.491522 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:45.491522 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 21:17:45.491522 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 21:17:45.491522 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 19 21:17:45.491522 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return ref.to_dict() May 19 21:17:45.491522 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 21:17:45.491522 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application next(self.gen) May 19 21:17:45.492792 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 21:17:45.492792 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application yield resource May 19 21:17:45.492792 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 21:17:45.492792 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application next(self.gen) May 19 21:17:45.492792 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 21:17:45.492792 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self.session.rollback() May 19 21:17:45.492792 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 21:17:45.492792 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self.force_reraise() May 19 21:17:45.492792 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 21:17:45.492792 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise self.value May 19 21:17:45.492792 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 21:17:45.492792 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 21:17:45.492792 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 21:17:45.492792 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application session.commit() May 19 21:17:45.492792 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 21:17:45.492792 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 21:17:45.492792 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 21:17:45.493798 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._prepare_impl() May 19 21:17:45.493798 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 21:17:45.493798 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self.session.flush() May 19 21:17:45.493798 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 21:17:45.493798 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._flush(objects) May 19 21:17:45.493798 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 21:17:45.493798 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 21:17:45.493798 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 21:17:45.493798 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application compat.raise_( May 19 21:17:45.493798 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 21:17:45.493798 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception May 19 21:17:45.493798 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 21:17:45.493798 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application flush_context.execute() May 19 21:17:45.493798 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 21:17:45.493798 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rec.execute(self) May 19 21:17:45.493798 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 21:17:45.493798 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 21:17:45.494721 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 21:17:45.494721 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 21:17:45.494721 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 21:17:45.494721 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 21:17:45.494721 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 21:17:45.494721 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 21:17:45.494721 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 21:17:45.494721 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 21:17:45.494721 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 21:17:45.494721 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 21:17:45.494721 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 21:17:45.494721 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 21:17:45.494721 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 21:17:45.494721 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 21:17:45.494721 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 21:17:45.494721 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception May 19 21:17:45.495557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 21:17:45.495557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 21:17:45.495557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 21:17:45.495557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 21:17:45.495557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 21:17:45.495557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application result = self._query(query) May 19 21:17:45.495557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 21:17:45.495557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application conn.query(q) May 19 21:17:45.495557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 21:17:45.495557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 21:17:45.495557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 21:17:45.495557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application result.read() May 19 21:17:45.495557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 21:17:45.495557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 21:17:45.495557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 21:17:45.495557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 21:17:45.496344 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 21:17:45.496344 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 21:17:45.496344 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 21:17:45.496344 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 21:17:45.496344 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 21:17:45.496344 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 21:17:45.496344 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [parameters: {'id': 'cc0e49ceac1e4dbfa80350d0e6ede415', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 21:17:45.496344 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 21:17:45.496344 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:45.496344 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:17:45.496344 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:45.496344 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:45.496344 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:45.496344 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:45.496344 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:45.496344 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:45.500902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:45.500902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:45.500902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:45.500902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:45.500902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:45.500902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:45.500902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 19 21:17:45.500902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 19 21:17:45.500902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:45.500902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:45.500902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 19 21:17:45.500902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 19 21:17:45.500902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 21:17:45.500902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 21:17:45.500902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 21:17:45.500902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:45.504683 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 233/477] 10.222.0.65 () {60 vars in 1131 bytes} [Thu May 19 21:17:45 2022] POST /identity/v3/roles => generated 144 bytes in 264 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:45.504683 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ba637-4eb0-45e3-9508-f968589536e7 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:45.508970 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:45.582835 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a4044-cae2-48cc-b24f-82192bd10eee None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:45.620679 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf41f4-f2c4-47fe-8dc8-723fd955fa01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:45.621465 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf41f4-f2c4-47fe-8dc8-723fd955fa01 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:45.621646 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf41f4-f2c4-47fe-8dc8-723fd955fa01 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:45.622008 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf41f4-f2c4-47fe-8dc8-723fd955fa01 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:45.622774 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbf41f4-f2c4-47fe-8dc8-723fd955fa01 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:45.624615 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbf41f4-f2c4-47fe-8dc8-723fd955fa01 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:45.632835 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:45.635540 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ba637-4eb0-45e3-9508-f968589536e7 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:45.656403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 245/478] 10.222.0.65 () {60 vars in 1132 bytes} [Thu May 19 21:17:45 2022] POST /identity/v3/users => generated 249 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:17:45.694503 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f409a97-8216-465f-8d37-f3bda7f2edb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:45.695396 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f409a97-8216-465f-8d37-f3bda7f2edb8 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:45.695678 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f409a97-8216-465f-8d37-f3bda7f2edb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:45.695952 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f409a97-8216-465f-8d37-f3bda7f2edb8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:45.720726 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f409a97-8216-465f-8d37-f3bda7f2edb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:45.729130 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-1f409a97-8216-465f-8d37-f3bda7f2edb8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:17:45.729130 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:45.729130 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:45.729130 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:45.729130 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:45.729130 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:45.729130 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:45.729130 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:45.729130 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:45.729130 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:45.729130 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:45.729130 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:45.729130 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:45.729130 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:45.729130 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:45.729130 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:45.730090 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:45.730090 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:45.730090 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:45.730090 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:45.730090 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:45.730090 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:45.730090 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:45.730090 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:45.730090 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:45.730090 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:45.730090 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:45.730090 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:45.730090 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:45.730090 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:45.730090 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:45.730090 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:45.730968 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:45.730968 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:45.730968 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:45.730968 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:45.730968 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:45.730968 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:45.730968 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:45.730968 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:45.730968 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:45.730968 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:45.730968 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 21:17:45.730968 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:45.738818 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 234/479] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 21:17:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:45.745116 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf41f4-f2c4-47fe-8dc8-723fd955fa01 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:45.912763 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf41f4-f2c4-47fe-8dc8-723fd955fa01 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:45.950109 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc66d3-8acd-45a2-9920-9e3e4d1e563e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:45.950768 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc66d3-8acd-45a2-9920-9e3e4d1e563e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:45.951056 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc66d3-8acd-45a2-9920-9e3e4d1e563e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:45.951334 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc66d3-8acd-45a2-9920-9e3e4d1e563e None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:45.952131 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc66d3-8acd-45a2-9920-9e3e4d1e563e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:45.952441 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc66d3-8acd-45a2-9920-9e3e4d1e563e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:45.953912 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc66d3-8acd-45a2-9920-9e3e4d1e563e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:45.964778 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 246/480] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:17:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:47.403187 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e43e8-3f5a-4ca3-ae08-28d1caa47ebb None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:47.404251 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e43e8-3f5a-4ca3-ae08-28d1caa47ebb None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:47.404730 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e43e8-3f5a-4ca3-ae08-28d1caa47ebb None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:47.406751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 235/481] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:47.412781 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d3163-ce70-47c1-8933-ca4226ad6cae None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:47.413837 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d3163-ce70-47c1-8933-ca4226ad6cae None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:47.414975 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d3163-ce70-47c1-8933-ca4226ad6cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:47.439991 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-fd9d3163-ce70-47c1-8933-ca4226ad6cae None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:47.441539 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fd9d3163-ce70-47c1-8933-ca4226ad6cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:47.522899 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:47.549042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 247/482] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:17:47.584785 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34e43e8-3f5a-4ca3-ae08-28d1caa47ebb None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:47.702695 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:47.794393 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34e43e8-3f5a-4ca3-ae08-28d1caa47ebb None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:47.885316 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf242ca-d114-4c1b-9f1a-b87c4d609141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:47.886778 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf242ca-d114-4c1b-9f1a-b87c4d609141 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:47.887271 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf242ca-d114-4c1b-9f1a-b87c4d609141 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:47.887747 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf242ca-d114-4c1b-9f1a-b87c4d609141 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:47.919648 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf242ca-d114-4c1b-9f1a-b87c4d609141 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:47.939544 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-fbf242ca-d114-4c1b-9f1a-b87c4d609141 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:17:47.939544 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:47.939544 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:47.939544 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:47.939544 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:47.939544 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:47.939544 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:47.939544 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:47.939544 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:47.939544 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:47.939544 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:47.939544 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:47.939544 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:47.939544 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:47.939544 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:47.939544 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:47.940573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:47.940573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:47.940573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:47.940573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:47.940573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:47.940573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:47.940573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:47.940573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:47.940573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:47.940573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:47.940573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:47.940573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:47.940573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:47.940573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:47.940573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:47.940573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:47.941368 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:47.941368 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:47.941368 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:47.941368 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:47.941368 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:47.941368 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:47.941368 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:47.941368 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:47.941368 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:47.941368 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:47.941368 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:17:47.941368 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:47.942488 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 236/483] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 21:17:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:47.948646 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d3163-ce70-47c1-8933-ca4226ad6cae None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:48.008974 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171ceb4-494c-4d01-a49b-c95936aff5b9 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:48.009393 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171ceb4-494c-4d01-a49b-c95936aff5b9 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:48.009909 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171ceb4-494c-4d01-a49b-c95936aff5b9 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:48.015258 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 237/484] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:17:48.021382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4b21f-1430-4889-80df-08a90d859ae7 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:48.021870 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4b21f-1430-4889-80df-08a90d859ae7 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:48.022370 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4b21f-1430-4889-80df-08a90d859ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:48.076069 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-a2d4b21f-1430-4889-80df-08a90d859ae7 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:48.078031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a2d4b21f-1430-4889-80df-08a90d859ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:48.165620 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d3163-ce70-47c1-8933-ca4226ad6cae None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:48.240084 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b4b1f5-0c0e-4fe2-8236-1c6eb4d7e433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:48.241579 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4b1f5-0c0e-4fe2-8236-1c6eb4d7e433 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:48.242777 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4b1f5-0c0e-4fe2-8236-1c6eb4d7e433 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:48.243281 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4b1f5-0c0e-4fe2-8236-1c6eb4d7e433 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:48.244247 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b4b1f5-0c0e-4fe2-8236-1c6eb4d7e433 None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:48.244743 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b4b1f5-0c0e-4fe2-8236-1c6eb4d7e433 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:48.246341 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b4b1f5-0c0e-4fe2-8236-1c6eb4d7e433 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:48.258455 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 248/485] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 21:17:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:48.264887 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b4b1f5-0c0e-4fe2-8236-1c6eb4d7e433 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:48.285841 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 238/486] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:17:48.316043 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d4b21f-1430-4889-80df-08a90d859ae7 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:48.356379 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:48.387667 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:48.437417 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d4b21f-1430-4889-80df-08a90d859ae7 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:48.456729 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b4b1f5-0c0e-4fe2-8236-1c6eb4d7e433 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:48.479213 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5c853b-1b5c-4574-b4fa-6276a8953be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:48.480906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c853b-1b5c-4574-b4fa-6276a8953be2 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:48.481388 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c853b-1b5c-4574-b4fa-6276a8953be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:48.482110 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c853b-1b5c-4574-b4fa-6276a8953be2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:48.508139 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5c853b-1b5c-4574-b4fa-6276a8953be2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:48.516061 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-3a5c853b-1b5c-4574-b4fa-6276a8953be2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:17:48.516061 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:48.516061 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:48.516061 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:48.516061 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:48.516061 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:48.516061 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:48.516061 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:48.516061 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:48.516061 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:48.516061 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:48.516061 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:48.516061 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:48.516061 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:48.516061 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:48.516061 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:48.516966 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:48.516966 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:48.516966 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:48.516966 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:48.516966 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:48.516966 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:48.516966 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:48.516966 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:48.516966 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:48.516966 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:48.516966 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:48.516966 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:48.516966 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:48.516966 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:48.516966 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:48.516966 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:48.517762 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:48.517762 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:48.517762 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:48.517762 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:48.517762 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:48.517762 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:48.517762 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:48.517762 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:48.517762 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:48.517762 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:48.517762 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:17:48.517762 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:48.517762 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 239/487] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 21:17:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:17:48.525239 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5c853b-1b5c-4574-b4fa-6276a8953be2 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:48.536563 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23cd900-7a8f-4366-8066-50c403c95d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:48.544983 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23cd900-7a8f-4366-8066-50c403c95d1f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:48.545568 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23cd900-7a8f-4366-8066-50c403c95d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:48.546057 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23cd900-7a8f-4366-8066-50c403c95d1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:48.566840 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23cd900-7a8f-4366-8066-50c403c95d1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:48.569559 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23cd900-7a8f-4366-8066-50c403c95d1f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:48.598661 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-a23cd900-7a8f-4366-8066-50c403c95d1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:48.598661 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:48.598661 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:48.598661 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:48.598661 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:48.598661 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:48.598661 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:48.598661 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:48.598661 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:48.598661 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:48.598661 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:48.598661 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:48.598661 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:17:48.598661 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:17:48.598661 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:17:48.599573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:17:48.599573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:48.599573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:48.599573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:48.599573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:48.599573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:48.599573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:48.599573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:48.599573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:17:48.599573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:48.599573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:48.599573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:48.599573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:48.599573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:48.599573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:48.599573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:48.599573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:48.600406 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:17:48.600406 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:17:48.600406 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:48.600406 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:17:48.603395 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 249/488] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:17:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:17:48.612771 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23cd900-7a8f-4366-8066-50c403c95d1f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:48.766349 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5c853b-1b5c-4574-b4fa-6276a8953be2 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:48.798166 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:48.837995 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:48.855410 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23cd900-7a8f-4366-8066-50c403c95d1f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:48.873096 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986654b-d4b4-46d1-abcb-7339604e105c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:48.874212 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986654b-d4b4-46d1-abcb-7339604e105c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:48.874724 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986654b-d4b4-46d1-abcb-7339604e105c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:48.875189 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986654b-d4b4-46d1-abcb-7339604e105c None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:48.876607 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5986654b-d4b4-46d1-abcb-7339604e105c None admin] RBAC: Adding query filter params (name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:48.877124 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5986654b-d4b4-46d1-abcb-7339604e105c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:48.879690 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5986654b-d4b4-46d1-abcb-7339604e105c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:48.922970 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 240/489] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 21:17:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:48.939356 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986654b-d4b4-46d1-abcb-7339604e105c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:48.995975 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de972c2-6600-4ac8-91c0-4a3f4b1974ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:48.997086 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de972c2-6600-4ac8-91c0-4a3f4b1974ad None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:48.997570 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de972c2-6600-4ac8-91c0-4a3f4b1974ad None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:48.998058 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de972c2-6600-4ac8-91c0-4a3f4b1974ad None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:48.999887 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de972c2-6600-4ac8-91c0-4a3f4b1974ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:49.000433 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de972c2-6600-4ac8-91c0-4a3f4b1974ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:49.003096 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de972c2-6600-4ac8-91c0-4a3f4b1974ad None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:49.027676 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 250/490] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:17:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:49.030058 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5038eeb-7abb-430d-a9d1-0a0b5bd4b61c None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:49.030589 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5038eeb-7abb-430d-a9d1-0a0b5bd4b61c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:49.031076 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5038eeb-7abb-430d-a9d1-0a0b5bd4b61c None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:49.033729 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 251/491] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:49.034898 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5038eeb-7abb-430d-a9d1-0a0b5bd4b61c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:49.171572 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986654b-d4b4-46d1-abcb-7339604e105c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:49.243433 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d504a-4cc9-4166-955b-9bc613fd5bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:49.244902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d504a-4cc9-4166-955b-9bc613fd5bec None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:49.245401 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d504a-4cc9-4166-955b-9bc613fd5bec None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:49.245883 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d504a-4cc9-4166-955b-9bc613fd5bec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:49.247279 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5038eeb-7abb-430d-a9d1-0a0b5bd4b61c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:49.268786 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d504a-4cc9-4166-955b-9bc613fd5bec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:49.271852 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d504a-4cc9-4166-955b-9bc613fd5bec None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:49.309210 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-413d504a-4cc9-4166-955b-9bc613fd5bec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:49.309210 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:49.309210 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:49.309210 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:49.309210 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:49.309210 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:49.309210 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:49.309210 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:49.309210 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:49.309210 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:49.309210 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:49.309210 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:49.309210 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:17:49.309210 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:17:49.309210 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:17:49.310876 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:17:49.310876 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:49.310876 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:49.310876 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:49.310876 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:49.310876 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:49.310876 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:49.310876 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:49.310876 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:49.310876 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:49.310876 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:49.310876 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:49.310876 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:49.310876 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:49.310876 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:49.310876 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:49.310876 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:49.312529 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:17:49.312529 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:17:49.312529 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:49.312529 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:49.312529 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 241/492] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:17:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:17:49.313010 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a2608-9b51-4158-add8-5efc154f199c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:49.313927 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a2608-9b51-4158-add8-5efc154f199c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:49.314165 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e380c8e-b3e0-44a9-90ae-51058d570f23 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:49.314412 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a2608-9b51-4158-add8-5efc154f199c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:49.314627 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e380c8e-b3e0-44a9-90ae-51058d570f23 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:49.314973 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e380c8e-b3e0-44a9-90ae-51058d570f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:49.318745 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a2608-9b51-4158-add8-5efc154f199c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:49.338095 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84a2608-9b51-4158-add8-5efc154f199c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:49.340995 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84a2608-9b51-4158-add8-5efc154f199c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:49.352613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-5e380c8e-b3e0-44a9-90ae-51058d570f23 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:49.354861 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5e380c8e-b3e0-44a9-90ae-51058d570f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:49.375183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-a84a2608-9b51-4158-add8-5efc154f199c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 21:17:49.375183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:49.375183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:49.375183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:49.375183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:49.375183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:49.375183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:49.375183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:49.375183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:49.375183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:49.375183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:49.375183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:49.375183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:17:49.375183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:17:49.375183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:17:49.375183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:17:49.377014 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:49.377014 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:49.377014 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:17:49.377014 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:17:49.377014 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:17:49.377014 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:17:49.377014 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:49.377014 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:49.377014 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:49.377014 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:49.377014 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:49.377014 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:17:49.377014 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:49.377014 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:49.377014 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:49.377014 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:49.377014 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:49.377014 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:49.380566 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:49.380566 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:49.380566 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:17:49.380566 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:17:49.380566 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:17:49.380566 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:17:49.380566 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:17:49.380566 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:17:49.380566 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 21:17:49.380566 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:17:49.380566 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 252/493] 10.222.0.65 () {56 vars in 1093 bytes} [Thu May 19 21:17:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:49.380566 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a2608-9b51-4158-add8-5efc154f199c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:49.393876 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:49.413881 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:49.461257 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 242/494] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:17:49.464864 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182de5a-ad07-4ffd-a440-bd035aa5357a None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:49.465650 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182de5a-ad07-4ffd-a440-bd035aa5357a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:49.466082 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182de5a-ad07-4ffd-a440-bd035aa5357a None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:49.468937 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 243/495] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:49.470120 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d182de5a-ad07-4ffd-a440-bd035aa5357a None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:49.583815 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a2608-9b51-4158-add8-5efc154f199c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:49.625891 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf8c9ab-effd-46f9-9f72-7c0de4ebf122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:49.626891 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8c9ab-effd-46f9-9f72-7c0de4ebf122 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:49.627290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8c9ab-effd-46f9-9f72-7c0de4ebf122 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:49.627683 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8c9ab-effd-46f9-9f72-7c0de4ebf122 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:49.628735 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf8c9ab-effd-46f9-9f72-7c0de4ebf122 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:49.629152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf8c9ab-effd-46f9-9f72-7c0de4ebf122 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:49.631036 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf8c9ab-effd-46f9-9f72-7c0de4ebf122 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:49.636626 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d182de5a-ad07-4ffd-a440-bd035aa5357a None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:49.650293 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 253/496] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:17:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:49.650293 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3190de2e-d32e-457d-9090-60908bba8ca7 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:49.650293 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3190de2e-d32e-457d-9090-60908bba8ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:49.650293 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3190de2e-d32e-457d-9090-60908bba8ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:49.675159 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-3190de2e-d32e-457d-9090-60908bba8ca7 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:49.677039 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3190de2e-d32e-457d-9090-60908bba8ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:49.706929 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d6a426-4897-4a25-b0c6-e1c061a324e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:49.708292 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6a426-4897-4a25-b0c6-e1c061a324e7 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:49.708832 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6a426-4897-4a25-b0c6-e1c061a324e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:49.709340 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6a426-4897-4a25-b0c6-e1c061a324e7 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:49.711879 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d6a426-4897-4a25-b0c6-e1c061a324e7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:49.712418 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d6a426-4897-4a25-b0c6-e1c061a324e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:49.715198 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d6a426-4897-4a25-b0c6-e1c061a324e7 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:49.782285 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 254/497] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:17:49.782285 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3190de2e-d32e-457d-9090-60908bba8ca7 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:49.791082 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 244/498] 10.222.0.65 () {56 vars in 1133 bytes} [Thu May 19 21:17:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:49.792558 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d6a426-4897-4a25-b0c6-e1c061a324e7 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:49.805476 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:49.828743 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:49.952872 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d6a426-4897-4a25-b0c6-e1c061a324e7 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:49.981008 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3190de2e-d32e-457d-9090-60908bba8ca7 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:49.994003 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f663fcc-5f0a-43f2-a418-cae407509a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:49.994870 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f663fcc-5f0a-43f2-a418-cae407509a63 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:49.995143 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f663fcc-5f0a-43f2-a418-cae407509a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:49.995414 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f663fcc-5f0a-43f2-a418-cae407509a63 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:50.003799 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f663fcc-5f0a-43f2-a418-cae407509a63 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:50.005293 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f663fcc-5f0a-43f2-a418-cae407509a63 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:50.023492 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-6f663fcc-5f0a-43f2-a418-cae407509a63 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 21:17:50.023492 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:50.023492 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:50.023492 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:50.023492 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:50.023492 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:50.023492 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:50.023492 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:50.023492 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:50.023492 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:50.023492 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:50.023492 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:50.023492 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:17:50.023492 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:17:50.023492 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:17:50.023492 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:17:50.025196 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:50.025196 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:50.025196 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:17:50.025196 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:17:50.025196 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:17:50.025196 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:17:50.025196 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:50.025196 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:50.025196 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:50.025196 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:50.025196 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:50.025196 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:50.025196 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:50.025196 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:50.025196 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:50.025196 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:50.025196 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:50.025196 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:50.026902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:50.026902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:50.026902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:17:50.026902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:17:50.026902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:17:50.026902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:17:50.026902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:17:50.026902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:17:50.026902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 21:17:50.026902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:50.026902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 245/499] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:17:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:50.026902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f663fcc-5f0a-43f2-a418-cae407509a63 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:50.050186 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94957cbc-50dc-4482-af92-3072d72433ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:50.051595 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94957cbc-50dc-4482-af92-3072d72433ff None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:50.052093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94957cbc-50dc-4482-af92-3072d72433ff None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:50.052546 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94957cbc-50dc-4482-af92-3072d72433ff None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:50.072872 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94957cbc-50dc-4482-af92-3072d72433ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:50.088305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-94957cbc-50dc-4482-af92-3072d72433ff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:17:50.088305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:50.088305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:50.088305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:50.088305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:50.088305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:50.088305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:50.088305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:50.088305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:50.088305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:50.088305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:50.088305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:50.088305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:50.088305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:50.088305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:50.088305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:50.090325 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:50.090325 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:50.090325 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:50.090325 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:50.090325 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:50.090325 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:50.090325 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:50.090325 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:50.090325 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:50.090325 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:50.090325 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:50.090325 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:17:50.090325 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:50.090325 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:50.090325 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:50.090325 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:50.091866 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:50.091866 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:50.091866 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:50.091866 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:50.091866 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:50.091866 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:50.091866 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:50.091866 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:50.091866 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:50.091866 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:50.091866 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:17:50.091866 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:17:50.091866 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 255/500] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 21:17:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:50.093152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94957cbc-50dc-4482-af92-3072d72433ff None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:50.116193 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:50.144581 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f663fcc-5f0a-43f2-a418-cae407509a63 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:50.183000 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d04a33-6b76-4b88-b20b-93193abcaafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:50.183898 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d04a33-6b76-4b88-b20b-93193abcaafd None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:50.184346 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d04a33-6b76-4b88-b20b-93193abcaafd None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:50.184792 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d04a33-6b76-4b88-b20b-93193abcaafd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:50.203212 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:50.247446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94957cbc-50dc-4482-af92-3072d72433ff None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:50.277866 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d04a33-6b76-4b88-b20b-93193abcaafd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:50.280444 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d04a33-6b76-4b88-b20b-93193abcaafd None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:50.292219 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1402210c-6bd9-4017-a0a1-180208a3ba04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:50.293546 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402210c-6bd9-4017-a0a1-180208a3ba04 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:50.294022 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402210c-6bd9-4017-a0a1-180208a3ba04 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:50.294515 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402210c-6bd9-4017-a0a1-180208a3ba04 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:50.312616 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1402210c-6bd9-4017-a0a1-180208a3ba04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:50.326256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-1402210c-6bd9-4017-a0a1-180208a3ba04 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:17:50.326256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:50.326256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:50.326256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:50.326256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:50.326256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:50.326256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:50.326256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:50.326256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:50.326256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:50.326256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:50.326256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:50.326256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:50.326256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:50.326256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:50.326256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:50.328039 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:50.328039 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:50.328039 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:50.328039 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:50.328039 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:50.328039 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:50.328039 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:50.328039 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:50.328039 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:50.328039 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:50.328039 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:50.328039 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:17:50.328039 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:50.328039 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:50.328039 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:50.328039 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:50.329225 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:50.329225 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:50.329225 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:50.329225 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:50.329225 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:50.329225 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:50.329225 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:50.329225 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:50.329225 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:50.329225 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:50.329225 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:17:50.329225 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:17:50.329225 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 256/501] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 21:17:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:50.330593 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5fac0d-97a2-4e8b-9f27-78416a14fbfb None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:50.330593 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5fac0d-97a2-4e8b-9f27-78416a14fbfb None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:50.330593 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5fac0d-97a2-4e8b-9f27-78416a14fbfb None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:50.338362 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 257/502] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:50.338362 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5fac0d-97a2-4e8b-9f27-78416a14fbfb None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:50.405134 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:50.442948 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 246/503] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:17:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:17:50.444211 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d04a33-6b76-4b88-b20b-93193abcaafd None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:50.485173 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5fac0d-97a2-4e8b-9f27-78416a14fbfb None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:50.530714 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:50.542458 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79bf649-6afe-4c0f-a6d1-8e19ebd63f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:50.543297 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bf649-6afe-4c0f-a6d1-8e19ebd63f39 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:50.543708 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bf649-6afe-4c0f-a6d1-8e19ebd63f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:50.544072 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79bf649-6afe-4c0f-a6d1-8e19ebd63f39 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:50.545029 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79bf649-6afe-4c0f-a6d1-8e19ebd63f39 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:50.545436 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79bf649-6afe-4c0f-a6d1-8e19ebd63f39 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:50.547074 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79bf649-6afe-4c0f-a6d1-8e19ebd63f39 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:50.608783 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d04a33-6b76-4b88-b20b-93193abcaafd None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:50.609628 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 258/504] 10.222.0.65 () {56 vars in 1137 bytes} [Thu May 19 21:17:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:50.610991 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79bf649-6afe-4c0f-a6d1-8e19ebd63f39 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:50.663286 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72db11d9-fd72-4ce4-a378-c0002c2cb75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:50.663993 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db11d9-fd72-4ce4-a378-c0002c2cb75c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:50.664270 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db11d9-fd72-4ce4-a378-c0002c2cb75c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:50.664540 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db11d9-fd72-4ce4-a378-c0002c2cb75c None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:50.665317 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72db11d9-fd72-4ce4-a378-c0002c2cb75c None admin] RBAC: Adding query filter params (name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:50.665624 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72db11d9-fd72-4ce4-a378-c0002c2cb75c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:50.667099 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72db11d9-fd72-4ce4-a378-c0002c2cb75c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:50.677153 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 247/505] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 21:17:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:50.678583 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ed231-e960-4fbe-b634-e90212af63ba None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:50.678863 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ed231-e960-4fbe-b634-e90212af63ba None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:50.679140 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ed231-e960-4fbe-b634-e90212af63ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:50.700187 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-9d7ed231-e960-4fbe-b634-e90212af63ba None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:50.702379 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9d7ed231-e960-4fbe-b634-e90212af63ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:50.747136 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:50.751115 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79bf649-6afe-4c0f-a6d1-8e19ebd63f39 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:50.799822 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 248/506] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:17:50.801272 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ed231-e960-4fbe-b634-e90212af63ba None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:50.811853 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2905937f-9b60-4ccb-aac2-a878292b3401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:50.813004 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905937f-9b60-4ccb-aac2-a878292b3401 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:50.814154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905937f-9b60-4ccb-aac2-a878292b3401 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:50.814638 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905937f-9b60-4ccb-aac2-a878292b3401 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:50.815955 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2905937f-9b60-4ccb-aac2-a878292b3401 None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:50.816502 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2905937f-9b60-4ccb-aac2-a878292b3401 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:50.818949 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2905937f-9b60-4ccb-aac2-a878292b3401 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:50.832030 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 259/507] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 21:17:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:50.833432 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2905937f-9b60-4ccb-aac2-a878292b3401 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:50.899291 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:50.954498 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ed231-e960-4fbe-b634-e90212af63ba None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:50.956175 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2905937f-9b60-4ccb-aac2-a878292b3401 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:51.003974 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfd3b2-661a-461b-948f-f23ffe18f691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:51.004385 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a8a82-880f-42a6-a773-0789cf5ee67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:51.005275 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfd3b2-661a-461b-948f-f23ffe18f691 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:51.005408 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a8a82-880f-42a6-a773-0789cf5ee67e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:51.005739 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfd3b2-661a-461b-948f-f23ffe18f691 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:51.005861 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a8a82-880f-42a6-a773-0789cf5ee67e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:51.006199 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfd3b2-661a-461b-948f-f23ffe18f691 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:51.006364 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a8a82-880f-42a6-a773-0789cf5ee67e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:51.026118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dfd3b2-661a-461b-948f-f23ffe18f691 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:51.028586 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dfd3b2-661a-461b-948f-f23ffe18f691 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:51.061436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-42dfd3b2-661a-461b-948f-f23ffe18f691 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:51.061436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:51.061436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:51.061436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:51.061436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:51.061436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:51.061436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:51.061436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:51.061436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:51.061436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:51.061436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:51.061436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:51.061436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:17:51.061436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:17:51.061436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:17:51.062639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:17:51.062639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:51.062639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:51.062639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:51.062639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:51.062639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:51.062639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:51.062639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:51.062639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:51.062639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:51.062639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:51.062639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:51.062639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:51.062639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:51.062639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:51.062639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:51.062639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:51.063637 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:17:51.063637 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:17:51.063637 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:51.063637 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:51.063637 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 249/508] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:17:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:51.064945 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfd3b2-661a-461b-948f-f23ffe18f691 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:51.097505 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:51.132168 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a8a82-880f-42a6-a773-0789cf5ee67e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:51.134732 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a8a82-880f-42a6-a773-0789cf5ee67e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:51.213543 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:51.224434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfd3b2-661a-461b-948f-f23ffe18f691 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:51.277014 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 260/509] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:17:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:17:51.278026 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a8a82-880f-42a6-a773-0789cf5ee67e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:51.281740 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0bf7c-cdb7-4d89-82af-c4745bc8d4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:51.283194 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0bf7c-cdb7-4d89-82af-c4745bc8d4bd None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:51.283685 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0bf7c-cdb7-4d89-82af-c4745bc8d4bd None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:51.284175 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0bf7c-cdb7-4d89-82af-c4745bc8d4bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:51.302996 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f0bf7c-cdb7-4d89-82af-c4745bc8d4bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:51.306554 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f0bf7c-cdb7-4d89-82af-c4745bc8d4bd None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:51.330747 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-15f0bf7c-cdb7-4d89-82af-c4745bc8d4bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:51.330747 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:51.330747 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:51.330747 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:51.330747 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:51.330747 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:51.330747 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:51.330747 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:51.330747 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:51.330747 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:51.330747 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:51.330747 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:51.330747 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:17:51.330747 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:17:51.330747 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:17:51.331736 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:17:51.331736 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:51.331736 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:51.331736 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:51.331736 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:51.331736 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:51.331736 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:51.331736 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:51.331736 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:51.331736 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:51.331736 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:51.331736 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:51.331736 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:51.331736 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:51.331736 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:51.331736 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:51.331736 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:51.332717 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:17:51.332717 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:17:51.332717 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:51.332717 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:51.332717 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 250/510] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:17:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:51.332972 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0bf7c-cdb7-4d89-82af-c4745bc8d4bd None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:51.409347 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:51.410492 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a8a82-880f-42a6-a773-0789cf5ee67e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:51.446592 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0bf7c-cdb7-4d89-82af-c4745bc8d4bd None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:51.454497 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811d7eb-7fda-4350-86db-90177bca22f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:51.455739 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811d7eb-7fda-4350-86db-90177bca22f5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:51.456040 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811d7eb-7fda-4350-86db-90177bca22f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:51.456539 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811d7eb-7fda-4350-86db-90177bca22f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:51.475332 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0811d7eb-7fda-4350-86db-90177bca22f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:51.478148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0811d7eb-7fda-4350-86db-90177bca22f5 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:51.485764 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02287c0-2a82-4bf8-b282-013d4f3d5b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:51.486700 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02287c0-2a82-4bf8-b282-013d4f3d5b08 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:51.486978 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02287c0-2a82-4bf8-b282-013d4f3d5b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:51.487250 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02287c0-2a82-4bf8-b282-013d4f3d5b08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:51.493565 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:51.498405 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02287c0-2a82-4bf8-b282-013d4f3d5b08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:51.499892 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02287c0-2a82-4bf8-b282-013d4f3d5b08 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:51.510541 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-0811d7eb-7fda-4350-86db-90177bca22f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:51.510541 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:51.510541 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:51.510541 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:51.510541 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:51.510541 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:51.510541 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:51.510541 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:51.510541 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:51.510541 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:51.510541 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:51.510541 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:51.510541 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:17:51.510541 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:17:51.510541 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:17:51.511535 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:17:51.511535 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:51.511535 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:51.511535 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:51.511535 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:51.511535 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:51.511535 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:51.511535 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:51.511535 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:17:51.511535 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:51.511535 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:51.511535 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:51.511535 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:51.511535 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:51.511535 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:51.511535 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:51.511535 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:51.512555 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:17:51.512555 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:17:51.512555 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:51.512555 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:17:51.517392 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 261/511] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:17:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:51.518851 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811d7eb-7fda-4350-86db-90177bca22f5 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:51.520253 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-a02287c0-2a82-4bf8-b282-013d4f3d5b08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:51.520253 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:51.520253 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:51.520253 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:51.520253 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:51.520253 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:51.520253 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:51.520253 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:51.520253 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:51.520253 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:51.520253 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:51.520253 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:51.520253 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:17:51.520253 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:17:51.520253 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:17:51.521682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:17:51.521682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:51.521682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:51.521682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:51.521682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:51.521682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:51.521682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:51.521682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:51.521682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:51.521682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:51.521682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:51.521682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:51.521682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:51.521682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:51.521682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:51.521682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:51.521682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:51.523867 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:17:51.523867 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:17:51.523867 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:51.523867 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:51.523867 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 251/512] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:17:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:51.523867 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02287c0-2a82-4bf8-b282-013d4f3d5b08 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:51.644932 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02287c0-2a82-4bf8-b282-013d4f3d5b08 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:51.647905 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811d7eb-7fda-4350-86db-90177bca22f5 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:51.687778 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:51.698521 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9578d08b-bdf1-49ed-9b08-ed9ea92d9bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:51.699560 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578d08b-bdf1-49ed-9b08-ed9ea92d9bdb None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:51.699975 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578d08b-bdf1-49ed-9b08-ed9ea92d9bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:51.700412 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578d08b-bdf1-49ed-9b08-ed9ea92d9bdb None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:51.701372 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9578d08b-bdf1-49ed-9b08-ed9ea92d9bdb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:51.701783 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9578d08b-bdf1-49ed-9b08-ed9ea92d9bdb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:51.703422 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9578d08b-bdf1-49ed-9b08-ed9ea92d9bdb None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:51.703610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc6a21-fef3-4a73-83be-e0b032fab3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:51.704325 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc6a21-fef3-4a73-83be-e0b032fab3a8 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:51.704609 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc6a21-fef3-4a73-83be-e0b032fab3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:51.704891 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc6a21-fef3-4a73-83be-e0b032fab3a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:51.717901 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc6a21-fef3-4a73-83be-e0b032fab3a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:51.719386 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 262/513] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:17:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:51.719577 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc6a21-fef3-4a73-83be-e0b032fab3a8 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:51.720765 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9578d08b-bdf1-49ed-9b08-ed9ea92d9bdb None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:51.744561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-9abc6a21-fef3-4a73-83be-e0b032fab3a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:51.744561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:51.744561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:51.744561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:51.744561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:51.744561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:51.744561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:51.744561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:51.744561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:51.744561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:51.744561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:51.744561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:51.744561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:17:51.744561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:17:51.744561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:17:51.746320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:17:51.746320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:51.746320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:51.746320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:51.746320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:51.746320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:51.746320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:51.746320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:51.746320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:51.746320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:51.746320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:51.746320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:51.746320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:51.746320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:51.746320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:51.746320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:51.746320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:51.748259 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:17:51.748259 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:17:51.748259 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:51.748259 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:51.748259 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 252/514] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:17:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:51.748259 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc6a21-fef3-4a73-83be-e0b032fab3a8 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:51.801958 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:51.858467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9578d08b-bdf1-49ed-9b08-ed9ea92d9bdb None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:51.886280 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc6a21-fef3-4a73-83be-e0b032fab3a8 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:51.906022 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6516bd7e-6687-4446-a089-37c95c07422c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:51.906787 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6516bd7e-6687-4446-a089-37c95c07422c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:51.907079 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6516bd7e-6687-4446-a089-37c95c07422c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:51.907433 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6516bd7e-6687-4446-a089-37c95c07422c None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:51.908238 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6516bd7e-6687-4446-a089-37c95c07422c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:51.908550 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6516bd7e-6687-4446-a089-37c95c07422c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:51.910104 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6516bd7e-6687-4446-a089-37c95c07422c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:51.932510 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2166a-2b93-4e0b-b194-b26ded246a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:51.933203 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2166a-2b93-4e0b-b194-b26ded246a03 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:51.933486 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2166a-2b93-4e0b-b194-b26ded246a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:51.933761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2166a-2b93-4e0b-b194-b26ded246a03 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:51.934561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b2166a-2b93-4e0b-b194-b26ded246a03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:51.934873 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b2166a-2b93-4e0b-b194-b26ded246a03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:51.935214 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 263/515] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:17:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:51.936120 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6516bd7e-6687-4446-a089-37c95c07422c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:51.936455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b2166a-2b93-4e0b-b194-b26ded246a03 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:51.954006 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 253/516] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:17:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:51.954936 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2166a-2b93-4e0b-b194-b26ded246a03 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:52.000201 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:52.065454 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6516bd7e-6687-4446-a089-37c95c07422c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:52.087670 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2166a-2b93-4e0b-b194-b26ded246a03 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:52.089930 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:52.107136 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b30e920-defe-436d-8f91-efcad98543b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:52.108069 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30e920-defe-436d-8f91-efcad98543b4 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:52.108438 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30e920-defe-436d-8f91-efcad98543b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:52.108816 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30e920-defe-436d-8f91-efcad98543b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:52.109789 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b30e920-defe-436d-8f91-efcad98543b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:52.110189 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b30e920-defe-436d-8f91-efcad98543b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:52.111716 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b30e920-defe-436d-8f91-efcad98543b4 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:52.126098 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 264/517] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:17:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:52.127006 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b30e920-defe-436d-8f91-efcad98543b4 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:52.131180 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe136043-0f4c-4ade-9b92-af08af46dc13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:52.131992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe136043-0f4c-4ade-9b92-af08af46dc13 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:52.132329 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe136043-0f4c-4ade-9b92-af08af46dc13 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:52.132660 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe136043-0f4c-4ade-9b92-af08af46dc13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:52.140279 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe136043-0f4c-4ade-9b92-af08af46dc13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:52.141792 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe136043-0f4c-4ade-9b92-af08af46dc13 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:52.157185 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-fe136043-0f4c-4ade-9b92-af08af46dc13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:17:52.157185 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:52.157185 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:52.157185 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:52.157185 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:52.157185 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:52.157185 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:52.157185 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:52.157185 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:52.157185 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:52.157185 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:52.157185 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:52.157185 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:17:52.157185 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:17:52.157185 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:17:52.158141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:17:52.158141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:52.158141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:52.158141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:52.158141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:52.158141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:52.158141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:52.158141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:52.158141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:52.158141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:52.158141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:52.158141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:52.158141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:52.158141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:52.158141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:52.158141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:52.158141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:52.159020 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:17:52.159020 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:17:52.159020 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:17:52.159020 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:17:52.159020 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:17:52.159020 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:17:52.159020 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:17:52.159020 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:52.159020 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 254/518] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:17:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:52.159499 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe136043-0f4c-4ade-9b92-af08af46dc13 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:52.251184 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b30e920-defe-436d-8f91-efcad98543b4 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:52.256206 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:52.262353 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe136043-0f4c-4ade-9b92-af08af46dc13 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:52.286139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30809e7c-1b35-4ef5-9680-ffe031d65804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:52.286839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30809e7c-1b35-4ef5-9680-ffe031d65804 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:52.287125 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30809e7c-1b35-4ef5-9680-ffe031d65804 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:52.287400 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30809e7c-1b35-4ef5-9680-ffe031d65804 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:52.288138 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30809e7c-1b35-4ef5-9680-ffe031d65804 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:52.288581 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30809e7c-1b35-4ef5-9680-ffe031d65804 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:52.290676 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30809e7c-1b35-4ef5-9680-ffe031d65804 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:52.298845 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2c4ac-54f3-4bd7-979c-707e85853e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:52.299663 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2c4ac-54f3-4bd7-979c-707e85853e1a None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:52.299973 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2c4ac-54f3-4bd7-979c-707e85853e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:52.300252 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2c4ac-54f3-4bd7-979c-707e85853e1a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:52.302790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 265/519] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:17:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:52.303605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30809e7c-1b35-4ef5-9680-ffe031d65804 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:52.312745 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a2c4ac-54f3-4bd7-979c-707e85853e1a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:52.314229 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a2c4ac-54f3-4bd7-979c-707e85853e1a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:52.329992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-30a2c4ac-54f3-4bd7-979c-707e85853e1a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 21:17:52.329992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:52.329992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:52.329992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:52.329992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:52.329992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:52.329992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:52.329992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:52.329992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:52.329992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:52.329992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:52.329992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:52.329992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:17:52.329992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:17:52.329992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:17:52.329992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:17:52.330980 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:52.330980 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:52.330980 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:17:52.330980 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:17:52.330980 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:17:52.330980 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:17:52.330980 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:52.330980 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:52.330980 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:52.330980 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:52.330980 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:52.330980 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:52.330980 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:52.330980 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:52.330980 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:52.330980 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:52.330980 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:52.330980 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:52.331920 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:52.331920 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:52.331920 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:17:52.331920 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:17:52.331920 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:17:52.331920 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:17:52.331920 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:17:52.331920 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:17:52.331920 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 21:17:52.331920 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:52.331920 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 255/520] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:17:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:52.332494 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2c4ac-54f3-4bd7-979c-707e85853e1a None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:52.353823 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:52.415430 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30809e7c-1b35-4ef5-9680-ffe031d65804 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:52.437348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2c4ac-54f3-4bd7-979c-707e85853e1a None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:52.452396 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162fa8c2-cb78-451e-825c-84bcb1239f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:52.453081 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fa8c2-cb78-451e-825c-84bcb1239f40 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:52.453367 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fa8c2-cb78-451e-825c-84bcb1239f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:52.453647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fa8c2-cb78-451e-825c-84bcb1239f40 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:52.454502 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162fa8c2-cb78-451e-825c-84bcb1239f40 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:52.455902 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162fa8c2-cb78-451e-825c-84bcb1239f40 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:52.472993 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f353b4-8aa8-4cdc-bb3e-d6803fb860b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:52.473808 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f353b4-8aa8-4cdc-bb3e-d6803fb860b1 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:52.474084 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f353b4-8aa8-4cdc-bb3e-d6803fb860b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:52.474385 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f353b4-8aa8-4cdc-bb3e-d6803fb860b1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:52.477663 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 266/521] 10.222.0.65 () {60 vars in 1132 bytes} [Thu May 19 21:17:52 2022] POST /identity/v3/users => generated 247 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:17:52.478529 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162fa8c2-cb78-451e-825c-84bcb1239f40 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:52.482545 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f353b4-8aa8-4cdc-bb3e-d6803fb860b1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:52.484014 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f353b4-8aa8-4cdc-bb3e-d6803fb860b1 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:52.503260 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-75f353b4-8aa8-4cdc-bb3e-d6803fb860b1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 21:17:52.503260 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:52.503260 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:52.503260 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:52.503260 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:52.503260 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:52.503260 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:52.503260 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:52.503260 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:52.503260 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:52.503260 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:52.503260 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:52.503260 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:17:52.503260 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:17:52.503260 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:17:52.503260 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:17:52.504531 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:52.504531 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:52.504531 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:17:52.504531 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:17:52.504531 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:17:52.504531 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:17:52.504531 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:52.504531 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:52.504531 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:52.504531 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:52.504531 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:52.504531 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:52.504531 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:52.504531 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:52.504531 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:52.504531 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:52.504531 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:52.504531 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:52.505552 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:52.505552 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:52.505552 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:17:52.505552 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:17:52.505552 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:17:52.505552 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:17:52.505552 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:17:52.505552 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:17:52.505552 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 21:17:52.505552 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:52.505552 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 256/522] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 21:17:52 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:52.505552 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f353b4-8aa8-4cdc-bb3e-d6803fb860b1 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:52.527759 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:52.600944 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:52.610976 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162fa8c2-cb78-451e-825c-84bcb1239f40 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:52.629257 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f353b4-8aa8-4cdc-bb3e-d6803fb860b1 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:52.654888 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845922f-07cd-430f-b44c-51f210f5cac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:52.655904 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845922f-07cd-430f-b44c-51f210f5cac8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:52.656310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845922f-07cd-430f-b44c-51f210f5cac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:52.656605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845922f-07cd-430f-b44c-51f210f5cac8 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:52.657581 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2845922f-07cd-430f-b44c-51f210f5cac8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:52.657909 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2845922f-07cd-430f-b44c-51f210f5cac8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:52.660096 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2845922f-07cd-430f-b44c-51f210f5cac8 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:52.671482 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38eaff7-233e-4b70-ae48-436f369e96f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:52.672332 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38eaff7-233e-4b70-ae48-436f369e96f9 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:52.672607 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38eaff7-233e-4b70-ae48-436f369e96f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:52.672874 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38eaff7-233e-4b70-ae48-436f369e96f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:52.681272 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38eaff7-233e-4b70-ae48-436f369e96f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:52.682792 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38eaff7-233e-4b70-ae48-436f369e96f9 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:52.683418 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 267/523] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 21:17:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:52.684265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845922f-07cd-430f-b44c-51f210f5cac8 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:52.699273 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-c38eaff7-233e-4b70-ae48-436f369e96f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:17:52.699273 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:52.699273 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:52.699273 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:52.699273 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:52.699273 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:52.699273 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:52.699273 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:52.699273 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:52.699273 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:52.699273 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:52.699273 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:52.699273 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:17:52.699273 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:17:52.699273 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:17:52.700264 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:17:52.700264 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:52.700264 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:52.700264 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:52.700264 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:52.700264 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:52.700264 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:52.700264 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:52.700264 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:52.700264 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:52.700264 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:52.700264 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:52.700264 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:52.700264 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:52.700264 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:52.700264 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:52.700264 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:52.701093 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:17:52.701093 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:17:52.701093 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:17:52.701093 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:17:52.701093 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:17:52.701093 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:17:52.701093 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:17:52.701093 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:52.701093 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 257/524] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:17:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:52.701548 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38eaff7-233e-4b70-ae48-436f369e96f9 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:52.790625 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:52.806450 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845922f-07cd-430f-b44c-51f210f5cac8 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:52.814665 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38eaff7-233e-4b70-ae48-436f369e96f9 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:52.847245 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7843749d-5569-4d71-b580-eda6b508dec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:52.848187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843749d-5569-4d71-b580-eda6b508dec3 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:52.848527 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843749d-5569-4d71-b580-eda6b508dec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:52.848900 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843749d-5569-4d71-b580-eda6b508dec3 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:52.850516 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7843749d-5569-4d71-b580-eda6b508dec3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:52.850516 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7843749d-5569-4d71-b580-eda6b508dec3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:52.852175 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7843749d-5569-4d71-b580-eda6b508dec3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:52.854490 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c08b959-7561-430d-ba0e-a5cfe4fc05f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:52.855340 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08b959-7561-430d-ba0e-a5cfe4fc05f5 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:52.855642 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08b959-7561-430d-ba0e-a5cfe4fc05f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:52.855925 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08b959-7561-430d-ba0e-a5cfe4fc05f5 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:52.856810 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c08b959-7561-430d-ba0e-a5cfe4fc05f5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:52.857122 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c08b959-7561-430d-ba0e-a5cfe4fc05f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:52.858955 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c08b959-7561-430d-ba0e-a5cfe4fc05f5 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:52.875153 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 268/525] 10.222.0.65 () {56 vars in 1137 bytes} [Thu May 19 21:17:52 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:52.876353 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7843749d-5569-4d71-b580-eda6b508dec3 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:52.879608 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 258/526] 10.222.0.65 () {56 vars in 1139 bytes} [Thu May 19 21:17:52 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:52.880302 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c08b959-7561-430d-ba0e-a5cfe4fc05f5 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:52.896984 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:52.989071 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c08b959-7561-430d-ba0e-a5cfe4fc05f5 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:52.989275 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7843749d-5569-4d71-b580-eda6b508dec3 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:53.028510 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913f430-0ed2-46d1-823b-f3eaec9688ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:53.028759 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd3448-225a-48c2-a765-94070e8f2a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:53.029190 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913f430-0ed2-46d1-823b-f3eaec9688ec None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:53.029326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd3448-225a-48c2-a765-94070e8f2a62 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:53.029481 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913f430-0ed2-46d1-823b-f3eaec9688ec None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:53.029570 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd3448-225a-48c2-a765-94070e8f2a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:53.029734 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd3448-225a-48c2-a765-94070e8f2a62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:53.029823 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913f430-0ed2-46d1-823b-f3eaec9688ec None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:53.030562 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6913f430-0ed2-46d1-823b-f3eaec9688ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:53.030654 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecd3448-225a-48c2-a765-94070e8f2a62 None admin] RBAC: Adding query filter params (role.id=9dc26e65165b49d6916b54458f847755, scope.project.id=c4940befa9c4405699c72deedc8a2127, user.id=e254671b21fe4661a381b9521810cf80) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:53.030876 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6913f430-0ed2-46d1-823b-f3eaec9688ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:53.030966 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecd3448-225a-48c2-a765-94070e8f2a62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:53.032330 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6913f430-0ed2-46d1-823b-f3eaec9688ec None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:53.032604 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecd3448-225a-48c2-a765-94070e8f2a62 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:53.034415 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:53.041795 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 269/527] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:17:52 2022] GET /identity/v3/role_assignments?user.id=e254671b21fe4661a381b9521810cf80&scope.project.id=c4940befa9c4405699c72deedc8a2127&role.id=9dc26e65165b49d6916b54458f847755 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:53.046411 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd3448-225a-48c2-a765-94070e8f2a62 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:53.058680 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 259/528] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 21:17:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:53.064544 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913f430-0ed2-46d1-823b-f3eaec9688ec None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:53.159150 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:53.186395 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd3448-225a-48c2-a765-94070e8f2a62 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:53.187231 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913f430-0ed2-46d1-823b-f3eaec9688ec None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:53.229113 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259d1d9a-a2cd-4ab5-b0a1-325d9dfe51c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:53.229938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d1d9a-a2cd-4ab5-b0a1-325d9dfe51c8 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:53.230220 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d1d9a-a2cd-4ab5-b0a1-325d9dfe51c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:53.230520 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d1d9a-a2cd-4ab5-b0a1-325d9dfe51c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:53.233330 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec07ef5-9f15-4e9d-bcaf-6da07e49ee21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:53.235960 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec07ef5-9f15-4e9d-bcaf-6da07e49ee21 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:53.236445 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec07ef5-9f15-4e9d-bcaf-6da07e49ee21 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:53.236896 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec07ef5-9f15-4e9d-bcaf-6da07e49ee21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:53.311012 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:53.323544 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec07ef5-9f15-4e9d-bcaf-6da07e49ee21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:53.323767 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259d1d9a-a2cd-4ab5-b0a1-325d9dfe51c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:53.325013 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec07ef5-9f15-4e9d-bcaf-6da07e49ee21 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:53.325121 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259d1d9a-a2cd-4ab5-b0a1-325d9dfe51c8 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:53.417629 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:53.448909 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 270/529] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:17:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:17:53.450020 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec07ef5-9f15-4e9d-bcaf-6da07e49ee21 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:53.464618 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 260/530] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:17:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:17:53.465719 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259d1d9a-a2cd-4ab5-b0a1-325d9dfe51c8 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:53.568385 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec07ef5-9f15-4e9d-bcaf-6da07e49ee21 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:53.594609 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:53.605911 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259d1d9a-a2cd-4ab5-b0a1-325d9dfe51c8 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:53.609252 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc4a1f3-7bc0-48a6-a91e-1412fb1c600d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:53.610045 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4a1f3-7bc0-48a6-a91e-1412fb1c600d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:53.610487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4a1f3-7bc0-48a6-a91e-1412fb1c600d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:53.610860 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4a1f3-7bc0-48a6-a91e-1412fb1c600d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:53.611858 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc4a1f3-7bc0-48a6-a91e-1412fb1c600d None admin] RBAC: Adding query filter params (role.id=9dc26e65165b49d6916b54458f847755, scope.project.id=c4940befa9c4405699c72deedc8a2127, user.id=ea874ca4088d4e6c83c9d0497323cf6b) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:53.612267 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc4a1f3-7bc0-48a6-a91e-1412fb1c600d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:53.613898 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc4a1f3-7bc0-48a6-a91e-1412fb1c600d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:53.626559 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 271/531] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:17:53 2022] GET /identity/v3/role_assignments?user.id=ea874ca4088d4e6c83c9d0497323cf6b&scope.project.id=c4940befa9c4405699c72deedc8a2127&role.id=9dc26e65165b49d6916b54458f847755 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:53.627779 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc4a1f3-7bc0-48a6-a91e-1412fb1c600d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:53.655592 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38beb76d-00d9-4347-ba16-ee62406f03f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:53.656722 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38beb76d-00d9-4347-ba16-ee62406f03f2 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:53.657157 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38beb76d-00d9-4347-ba16-ee62406f03f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:53.657809 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38beb76d-00d9-4347-ba16-ee62406f03f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:53.673569 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38beb76d-00d9-4347-ba16-ee62406f03f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:53.675915 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38beb76d-00d9-4347-ba16-ee62406f03f2 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:53.703591 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-38beb76d-00d9-4347-ba16-ee62406f03f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:53.703591 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:53.703591 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:53.703591 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:53.703591 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:53.703591 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:53.703591 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:53.703591 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:53.703591 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:53.703591 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:53.703591 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:53.703591 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:53.703591 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:17:53.703591 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:17:53.703591 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:17:53.704773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:17:53.704773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:53.704773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:53.704773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:53.704773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:53.704773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:53.704773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:53.704773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:53.704773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:53.704773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:53.704773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:53.704773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:53.704773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:53.704773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:53.704773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:53.704773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:53.704773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:53.705852 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:17:53.705852 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:17:53.705852 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:53.705852 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:53.720385 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 261/532] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:17:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:53.721631 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38beb76d-00d9-4347-ba16-ee62406f03f2 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:53.729130 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:53.885414 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc4a1f3-7bc0-48a6-a91e-1412fb1c600d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:53.897385 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38beb76d-00d9-4347-ba16-ee62406f03f2 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:53.924435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7d009c-10b2-4c7e-afea-ebac536b3923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:53.925385 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d009c-10b2-4c7e-afea-ebac536b3923 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:53.925618 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d009c-10b2-4c7e-afea-ebac536b3923 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:53.925894 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d009c-10b2-4c7e-afea-ebac536b3923 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:53.938278 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7d009c-10b2-4c7e-afea-ebac536b3923 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:53.939912 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7d009c-10b2-4c7e-afea-ebac536b3923 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:53.940174 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afec18-a31d-4694-ae20-4ff1bd9c6322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:53.941013 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afec18-a31d-4694-ae20-4ff1bd9c6322 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:53.941795 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afec18-a31d-4694-ae20-4ff1bd9c6322 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:53.942227 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afec18-a31d-4694-ae20-4ff1bd9c6322 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:53.947651 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25afec18-a31d-4694-ae20-4ff1bd9c6322 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:53.948179 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25afec18-a31d-4694-ae20-4ff1bd9c6322 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:53.949857 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25afec18-a31d-4694-ae20-4ff1bd9c6322 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:53.959121 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-1c7d009c-10b2-4c7e-afea-ebac536b3923 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:53.959121 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:53.959121 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:53.959121 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:53.959121 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:53.959121 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:53.959121 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:53.959121 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:53.959121 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:53.959121 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:53.959121 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:53.959121 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:53.959121 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:17:53.959121 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:17:53.959121 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:17:53.960042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:17:53.960042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:53.960042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:53.960042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:53.960042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:53.960042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:53.960042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:53.960042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:53.960042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:17:53.960042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:53.960042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:53.960042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:53.960042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:53.960042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:53.960042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:53.960042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:53.960042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:53.961132 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:17:53.961132 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:17:53.961132 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:53.961132 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:17:53.961132 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 272/533] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:17:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:53.969170 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7d009c-10b2-4c7e-afea-ebac536b3923 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:53.973997 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 262/534] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:17:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:53.983149 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afec18-a31d-4694-ae20-4ff1bd9c6322 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:53.993213 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:54.140865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:54.152187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7d009c-10b2-4c7e-afea-ebac536b3923 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:54.168176 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afec18-a31d-4694-ae20-4ff1bd9c6322 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:54.214129 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83bc15f-6981-4485-a9be-8cb5944a886c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:54.214729 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83bc15f-6981-4485-a9be-8cb5944a886c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:54.215003 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83bc15f-6981-4485-a9be-8cb5944a886c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:54.215279 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83bc15f-6981-4485-a9be-8cb5944a886c None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:54.216015 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83bc15f-6981-4485-a9be-8cb5944a886c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:54.216322 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83bc15f-6981-4485-a9be-8cb5944a886c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:54.217805 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83bc15f-6981-4485-a9be-8cb5944a886c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:54.234642 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6e355-8254-4be0-85a9-3db3dbadeeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:54.236344 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6e355-8254-4be0-85a9-3db3dbadeeda None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:54.237122 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6e355-8254-4be0-85a9-3db3dbadeeda None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:54.237886 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6e355-8254-4be0-85a9-3db3dbadeeda None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:54.249485 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 273/535] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:17:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:17:54.252001 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b6e355-8254-4be0-85a9-3db3dbadeeda None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:54.255111 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83bc15f-6981-4485-a9be-8cb5944a886c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:54.255790 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b6e355-8254-4be0-85a9-3db3dbadeeda None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:54.286001 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-36b6e355-8254-4be0-85a9-3db3dbadeeda None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:17:54.286001 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:54.286001 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:54.286001 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:54.286001 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:54.286001 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:54.286001 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:54.286001 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:54.286001 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:54.286001 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:54.286001 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:54.286001 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:54.286001 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:17:54.286001 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:17:54.286001 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:17:54.287694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:17:54.287694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:54.287694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:54.287694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:54.287694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:54.287694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:54.287694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:54.287694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:54.287694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:54.287694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:54.287694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:54.287694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:54.287694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:54.287694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:54.287694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:54.287694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:54.287694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:54.288930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:17:54.288930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:17:54.288930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:17:54.288930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:17:54.288930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:17:54.288930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:17:54.288930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:17:54.288930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:54.292541 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 263/536] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:17:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:54.301361 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6e355-8254-4be0-85a9-3db3dbadeeda None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:54.434867 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:54.479103 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83bc15f-6981-4485-a9be-8cb5944a886c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:54.509183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6e355-8254-4be0-85a9-3db3dbadeeda None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:54.517673 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5003a070-e1ce-48cc-95db-8017e5a1eec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:54.518893 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003a070-e1ce-48cc-95db-8017e5a1eec1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:54.520309 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003a070-e1ce-48cc-95db-8017e5a1eec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:54.520802 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003a070-e1ce-48cc-95db-8017e5a1eec1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:54.535826 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5003a070-e1ce-48cc-95db-8017e5a1eec1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:54.537649 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5003a070-e1ce-48cc-95db-8017e5a1eec1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:54.571105 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-5003a070-e1ce-48cc-95db-8017e5a1eec1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:17:54.571105 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:54.571105 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:54.571105 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:54.571105 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:54.571105 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:54.571105 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:54.571105 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:54.571105 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:54.571105 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:54.571105 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:54.571105 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:54.571105 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:17:54.571105 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:17:54.571105 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:17:54.572033 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:17:54.572033 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:54.572033 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:54.572033 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:54.572033 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:54.572033 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:54.572033 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:54.572033 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:54.572033 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:17:54.572033 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:54.572033 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:54.572033 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:54.572033 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:54.572033 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:54.572033 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:54.572033 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:54.572033 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:54.572863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:17:54.572863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:17:54.572863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:17:54.572863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:17:54.572863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:17:54.572863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:17:54.572863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:17:54.572863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:17:54.576167 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 274/537] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:17:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:17:54.581676 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5003a070-e1ce-48cc-95db-8017e5a1eec1 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:54.589784 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291515d-600f-4fc3-b5f7-5c4924c756c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:54.590896 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291515d-600f-4fc3-b5f7-5c4924c756c8 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:54.591392 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291515d-600f-4fc3-b5f7-5c4924c756c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:54.591882 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291515d-600f-4fc3-b5f7-5c4924c756c8 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:54.592899 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291515d-600f-4fc3-b5f7-5c4924c756c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:54.593413 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291515d-600f-4fc3-b5f7-5c4924c756c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:54.595126 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291515d-600f-4fc3-b5f7-5c4924c756c8 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:54.626359 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:54.633408 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 264/538] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 21:17:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:54.637453 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291515d-600f-4fc3-b5f7-5c4924c756c8 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:54.820698 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5003a070-e1ce-48cc-95db-8017e5a1eec1 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:54.903467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95687fc-d5fc-4b57-bbc5-2d43996dd0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:54.904197 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95687fc-d5fc-4b57-bbc5-2d43996dd0e2 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:54.904485 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95687fc-d5fc-4b57-bbc5-2d43996dd0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:54.904991 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95687fc-d5fc-4b57-bbc5-2d43996dd0e2 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:54.905887 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95687fc-d5fc-4b57-bbc5-2d43996dd0e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:54.906214 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95687fc-d5fc-4b57-bbc5-2d43996dd0e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:54.907774 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95687fc-d5fc-4b57-bbc5-2d43996dd0e2 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:54.934773 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 275/539] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 21:17:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:17:54.940685 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95687fc-d5fc-4b57-bbc5-2d43996dd0e2 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:54.944094 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291515d-600f-4fc3-b5f7-5c4924c756c8 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:54.982622 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:55.002173 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e762fdb3-62cb-4e52-a0fb-9586db30f92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:55.002875 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762fdb3-62cb-4e52-a0fb-9586db30f92f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:55.003160 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762fdb3-62cb-4e52-a0fb-9586db30f92f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:55.003432 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762fdb3-62cb-4e52-a0fb-9586db30f92f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:55.004275 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e762fdb3-62cb-4e52-a0fb-9586db30f92f None admin] RBAC: Adding query filter params (role.id=9dc26e65165b49d6916b54458f847755, scope.project.id=c4940befa9c4405699c72deedc8a2127, user.id=51852131d4e5448f8152a51159bd78e4) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:55.004584 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e762fdb3-62cb-4e52-a0fb-9586db30f92f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:55.006048 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e762fdb3-62cb-4e52-a0fb-9586db30f92f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:55.020923 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 265/540] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:17:54 2022] GET /identity/v3/role_assignments?user.id=51852131d4e5448f8152a51159bd78e4&scope.project.id=c4940befa9c4405699c72deedc8a2127&role.id=9dc26e65165b49d6916b54458f847755 => generated 259 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:55.085282 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57769dae-01fc-40c9-805d-2c35a6e14bdd None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:55.085538 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57769dae-01fc-40c9-805d-2c35a6e14bdd None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:55.086505 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57769dae-01fc-40c9-805d-2c35a6e14bdd None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:55.088184 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 266/541] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:17:55.094728 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df2343-9f16-40c1-85c9-09bd2eeed9cc None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:55.095368 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df2343-9f16-40c1-85c9-09bd2eeed9cc None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:55.095851 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df2343-9f16-40c1-85c9-09bd2eeed9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:55.103704 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:55.118827 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-70df2343-9f16-40c1-85c9-09bd2eeed9cc None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:55.120479 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-70df2343-9f16-40c1-85c9-09bd2eeed9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:55.156728 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95687fc-d5fc-4b57-bbc5-2d43996dd0e2 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:55.243794 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc545179-2c15-406a-9a70-628708ba0b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:55.244984 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc545179-2c15-406a-9a70-628708ba0b37 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:55.245515 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc545179-2c15-406a-9a70-628708ba0b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:55.246043 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc545179-2c15-406a-9a70-628708ba0b37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:55.247199 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc545179-2c15-406a-9a70-628708ba0b37 None admin] RBAC: Adding query filter params (role.id=9dc26e65165b49d6916b54458f847755, scope.project.id=c4940befa9c4405699c72deedc8a2127, user.id=2201e66caa4b4a57be09d845d7cb0023) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:55.248912 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc545179-2c15-406a-9a70-628708ba0b37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:55.250794 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc545179-2c15-406a-9a70-628708ba0b37 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:55.257092 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 267/542] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:17:55.266233 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 276/543] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:17:54 2022] GET /identity/v3/role_assignments?user.id=2201e66caa4b4a57be09d845d7cb0023&scope.project.id=c4940befa9c4405699c72deedc8a2127&role.id=9dc26e65165b49d6916b54458f847755 => generated 259 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:17:55.288100 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70df2343-9f16-40c1-85c9-09bd2eeed9cc None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:55.423055 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70df2343-9f16-40c1-85c9-09bd2eeed9cc None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:55.459817 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:55.485108 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999693b7-5995-4845-8b90-2c0ef6656aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:55.485956 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999693b7-5995-4845-8b90-2c0ef6656aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:55.486262 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999693b7-5995-4845-8b90-2c0ef6656aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:55.486640 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999693b7-5995-4845-8b90-2c0ef6656aa1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:55.510134 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999693b7-5995-4845-8b90-2c0ef6656aa1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:55.518773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-999693b7-5995-4845-8b90-2c0ef6656aa1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:17:55.518773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:55.518773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:55.518773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:55.518773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:55.518773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:55.518773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:55.518773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:55.518773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:55.518773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:55.518773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:55.518773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:55.518773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:55.518773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:55.518773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:55.518773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:55.519780 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:55.519780 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:55.519780 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:55.519780 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:55.519780 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:55.519780 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:55.519780 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:55.519780 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:55.519780 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:55.519780 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:55.519780 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:55.519780 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:55.519780 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:55.519780 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:55.519780 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:55.519780 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:55.521038 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:55.521038 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:55.521038 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:55.521038 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:55.521038 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:55.521038 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:55.521038 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:55.521038 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:55.521038 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:55.521038 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:55.521038 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:17:55.521038 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:55.521038 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 268/544] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 21:17:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:55.527035 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc545179-2c15-406a-9a70-628708ba0b37 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:55.666786 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc545179-2c15-406a-9a70-628708ba0b37 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:55.721367 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1518c9-7c67-48d1-ba5a-2bf478139134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:55.722819 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1518c9-7c67-48d1-ba5a-2bf478139134 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:55.723522 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1518c9-7c67-48d1-ba5a-2bf478139134 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:55.724339 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1518c9-7c67-48d1-ba5a-2bf478139134 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:55.725614 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1518c9-7c67-48d1-ba5a-2bf478139134 None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:55.726212 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1518c9-7c67-48d1-ba5a-2bf478139134 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:55.727926 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1518c9-7c67-48d1-ba5a-2bf478139134 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:55.740989 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f213cdc9-d74c-4b2e-a538-d2155b886d68 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:55.741314 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f213cdc9-d74c-4b2e-a538-d2155b886d68 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:55.742130 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f213cdc9-d74c-4b2e-a538-d2155b886d68 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:55.743321 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 269/545] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:55.745613 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 277/546] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 21:17:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:55.746599 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:55.748860 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f13339-6b4a-4e1d-96a1-39bf98b209bc None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:55.749176 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f13339-6b4a-4e1d-96a1-39bf98b209bc None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:55.749503 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f13339-6b4a-4e1d-96a1-39bf98b209bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:55.759327 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1518c9-7c67-48d1-ba5a-2bf478139134 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:55.775360 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-08f13339-6b4a-4e1d-96a1-39bf98b209bc None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:55.778564 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-08f13339-6b4a-4e1d-96a1-39bf98b209bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:55.890951 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 270/547] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:17:55.922492 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1518c9-7c67-48d1-ba5a-2bf478139134 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:55.925497 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f13339-6b4a-4e1d-96a1-39bf98b209bc None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:55.963210 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b0fc5-f5df-44e7-8126-4e435953d712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:55.964049 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b0fc5-f5df-44e7-8126-4e435953d712 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:55.964341 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b0fc5-f5df-44e7-8126-4e435953d712 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:55.964621 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b0fc5-f5df-44e7-8126-4e435953d712 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:55.990310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6b0fc5-f5df-44e7-8126-4e435953d712 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:55.992234 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6b0fc5-f5df-44e7-8126-4e435953d712 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:56.027012 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-2b6b0fc5-f5df-44e7-8126-4e435953d712 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:56.027012 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:56.027012 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:56.027012 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:56.027012 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:56.027012 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:56.027012 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:56.027012 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:56.027012 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:56.027012 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:56.027012 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:56.027012 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:56.027012 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:17:56.027012 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:17:56.027012 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:17:56.029226 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:17:56.029226 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:56.029226 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:56.029226 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:56.029226 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:56.029226 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:56.029226 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:56.029226 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:56.029226 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:17:56.029226 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:56.029226 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:56.029226 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:56.029226 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:56.029226 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:56.029226 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:56.029226 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:56.029226 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:56.030668 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:17:56.030668 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:17:56.030668 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:56.030668 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:17:56.030668 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 278/548] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:17:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:17:56.038781 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b0fc5-f5df-44e7-8126-4e435953d712 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:56.105852 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f13339-6b4a-4e1d-96a1-39bf98b209bc None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:56.118651 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:56.146670 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434190aa-81b6-43ef-b9f3-f16308d42f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:56.148450 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434190aa-81b6-43ef-b9f3-f16308d42f16 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:56.148810 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434190aa-81b6-43ef-b9f3-f16308d42f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:56.149092 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434190aa-81b6-43ef-b9f3-f16308d42f16 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:56.161068 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434190aa-81b6-43ef-b9f3-f16308d42f16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:56.174610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-434190aa-81b6-43ef-b9f3-f16308d42f16 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:17:56.174610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:56.174610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:56.174610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:56.174610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:56.174610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:56.174610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:56.174610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:56.174610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:56.174610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:56.174610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:56.174610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:56.174610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:56.174610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:56.174610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:56.174610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:56.176075 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:56.176075 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:56.176075 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:56.176075 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:56.176075 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:56.176075 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:56.176075 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:56.176075 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:56.176075 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:56.176075 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:56.176075 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:56.176075 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:56.176075 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:56.176075 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:56.176075 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:56.176075 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:56.177047 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:56.177047 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:56.177047 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:56.177047 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:56.177047 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:56.177047 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:56.177047 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:56.177047 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:56.177047 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:56.177047 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:56.177047 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:17:56.177047 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:56.177047 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 271/549] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 21:17:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:17:56.183250 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434190aa-81b6-43ef-b9f3-f16308d42f16 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:56.209380 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:56.285368 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b0fc5-f5df-44e7-8126-4e435953d712 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:56.326332 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814cd539-2633-46a6-bde6-1b0f2e7d27e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:56.327074 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814cd539-2633-46a6-bde6-1b0f2e7d27e4 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:56.327365 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814cd539-2633-46a6-bde6-1b0f2e7d27e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:56.327650 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814cd539-2633-46a6-bde6-1b0f2e7d27e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:56.328634 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814cd539-2633-46a6-bde6-1b0f2e7d27e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:56.329091 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814cd539-2633-46a6-bde6-1b0f2e7d27e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:56.330661 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814cd539-2633-46a6-bde6-1b0f2e7d27e4 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:56.344510 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 279/550] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:56.350704 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814cd539-2633-46a6-bde6-1b0f2e7d27e4 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:56.456479 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434190aa-81b6-43ef-b9f3-f16308d42f16 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:56.467540 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814cd539-2633-46a6-bde6-1b0f2e7d27e4 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:56.484699 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:56.512136 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2c6b6-cf4b-46d8-b070-349f6dcf8307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:56.512836 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2c6b6-cf4b-46d8-b070-349f6dcf8307 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:56.513121 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2c6b6-cf4b-46d8-b070-349f6dcf8307 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:56.513397 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2c6b6-cf4b-46d8-b070-349f6dcf8307 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:56.514153 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c2c6b6-cf4b-46d8-b070-349f6dcf8307 None admin] RBAC: Adding query filter params (name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:56.514487 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c2c6b6-cf4b-46d8-b070-349f6dcf8307 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:56.514757 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb72682-659f-40a9-b34e-d8383559f8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:56.515661 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb72682-659f-40a9-b34e-d8383559f8db None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:56.515901 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c2c6b6-cf4b-46d8-b070-349f6dcf8307 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:56.516132 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb72682-659f-40a9-b34e-d8383559f8db None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:56.516506 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb72682-659f-40a9-b34e-d8383559f8db None admin] PATH_INFO: `/v3/users/swift` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:56.525728 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 272/551] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 21:17:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:56.526299 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb72682-659f-40a9-b34e-d8383559f8db None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:56.527231 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b2b9a-be99-49ce-ac27-c505b6fad5d3 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:56.527519 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b2b9a-be99-49ce-ac27-c505b6fad5d3 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:56.527796 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b2b9a-be99-49ce-ac27-c505b6fad5d3 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:56.528166 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb72682-659f-40a9-b34e-d8383559f8db None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:56.529324 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 273/552] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:56.534495 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d815a0-4e2c-4ca5-9a26-313e7e07ad12 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:56.534865 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d815a0-4e2c-4ca5-9a26-313e7e07ad12 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:56.535215 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d815a0-4e2c-4ca5-9a26-313e7e07ad12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:56.545415 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-9fb72682-659f-40a9-b34e-d8383559f8db None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 21:17:56.545415 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:56.545415 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:56.545415 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:56.545415 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:56.545415 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:56.545415 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:56.545415 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:56.545415 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:56.545415 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:56.545415 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:56.545415 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:56.545415 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:17:56.545415 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:17:56.545415 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:17:56.545415 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:17:56.546547 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:56.546547 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:56.546547 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:17:56.546547 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:17:56.546547 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:17:56.546547 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:17:56.546547 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:56.546547 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:56.546547 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:56.546547 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:56.546547 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:56.546547 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:17:56.546547 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:56.546547 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:56.546547 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:56.546547 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:56.546547 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:56.546547 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:56.547500 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:56.547500 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:56.547500 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:17:56.547500 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:17:56.547500 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:17:56.547500 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:17:56.547500 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:17:56.547500 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:17:56.547500 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 21:17:56.547500 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:17:56.553383 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 280/553] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 21:17:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:17:56.553383 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb72682-659f-40a9-b34e-d8383559f8db None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:56.560648 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-97d815a0-4e2c-4ca5-9a26-313e7e07ad12 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:56.561858 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-97d815a0-4e2c-4ca5-9a26-313e7e07ad12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:56.595826 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:56.657768 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 274/554] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:17:56.659169 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d815a0-4e2c-4ca5-9a26-313e7e07ad12 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:56.694333 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb72682-659f-40a9-b34e-d8383559f8db None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:56.735056 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be52fcb-f00b-4401-b40c-4bb39c4de82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:56.735889 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be52fcb-f00b-4401-b40c-4bb39c4de82d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:56.736181 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be52fcb-f00b-4401-b40c-4bb39c4de82d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:56.736549 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be52fcb-f00b-4401-b40c-4bb39c4de82d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:56.750526 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be52fcb-f00b-4401-b40c-4bb39c4de82d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:56.751997 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be52fcb-f00b-4401-b40c-4bb39c4de82d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:56.772042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-7be52fcb-f00b-4401-b40c-4bb39c4de82d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:56.772042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:56.772042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:56.772042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:56.772042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:56.772042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:56.772042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:56.772042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:56.772042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:56.772042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:56.772042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:56.772042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:56.772042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:17:56.772042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:17:56.772042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:17:56.773265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:17:56.773265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:56.773265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:56.773265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:56.773265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:56.773265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:56.773265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:56.773265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:56.773265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:17:56.773265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:56.773265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:56.773265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:56.773265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:56.773265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:56.773265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:56.773265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:56.773265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:56.774171 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:17:56.774171 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:17:56.774171 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:56.774171 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:17:56.779072 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 281/555] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:17:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:56.780130 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be52fcb-f00b-4401-b40c-4bb39c4de82d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:56.782934 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d815a0-4e2c-4ca5-9a26-313e7e07ad12 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:56.809322 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:56.823436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7afbf9-03f7-40e2-98ed-8095de09fd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:56.824124 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7afbf9-03f7-40e2-98ed-8095de09fd27 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:56.824407 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7afbf9-03f7-40e2-98ed-8095de09fd27 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:56.824686 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7afbf9-03f7-40e2-98ed-8095de09fd27 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:56.825530 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7afbf9-03f7-40e2-98ed-8095de09fd27 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:56.825838 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7afbf9-03f7-40e2-98ed-8095de09fd27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:56.827310 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7afbf9-03f7-40e2-98ed-8095de09fd27 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:56.862291 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 275/556] 10.222.0.65 () {56 vars in 1135 bytes} [Thu May 19 21:17:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:56.863587 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7afbf9-03f7-40e2-98ed-8095de09fd27 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:56.888616 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:56.909903 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be52fcb-f00b-4401-b40c-4bb39c4de82d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:56.952078 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8cf86-d056-4fab-b0ef-2f1c7c4ed42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:56.953701 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8cf86-d056-4fab-b0ef-2f1c7c4ed42a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:56.954149 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8cf86-d056-4fab-b0ef-2f1c7c4ed42a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:56.954601 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8cf86-d056-4fab-b0ef-2f1c7c4ed42a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:56.967744 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc8cf86-d056-4fab-b0ef-2f1c7c4ed42a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:56.977199 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-ecc8cf86-d056-4fab-b0ef-2f1c7c4ed42a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:17:56.977199 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:56.977199 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:56.977199 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:56.977199 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:56.977199 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:56.977199 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:56.977199 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:56.977199 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:56.977199 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:56.977199 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:56.977199 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:56.977199 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:56.977199 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:56.977199 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:56.977199 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:56.978361 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:56.978361 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:56.978361 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:56.978361 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:56.978361 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:56.978361 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:56.978361 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:56.978361 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:56.978361 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:56.978361 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:56.978361 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:56.978361 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:17:56.978361 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:56.978361 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:56.978361 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:56.978361 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:56.979618 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:56.979618 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:56.979618 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:56.979618 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:56.979618 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:56.979618 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:56.979618 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:56.979618 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:56.979618 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:56.979618 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:56.979618 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:17:56.979618 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:17:56.980247 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7afbf9-03f7-40e2-98ed-8095de09fd27 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:56.987238 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 282/557] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 21:17:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:56.988503 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8cf86-d056-4fab-b0ef-2f1c7c4ed42a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:57.017630 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b1dd1-8440-4b61-a338-4f2901642899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:57.018519 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b1dd1-8440-4b61-a338-4f2901642899 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:57.018868 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b1dd1-8440-4b61-a338-4f2901642899 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:57.019230 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b1dd1-8440-4b61-a338-4f2901642899 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:57.020079 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61b1dd1-8440-4b61-a338-4f2901642899 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:57.020449 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61b1dd1-8440-4b61-a338-4f2901642899 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:57.021896 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61b1dd1-8440-4b61-a338-4f2901642899 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:57.038312 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 276/558] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:57.038312 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b1dd1-8440-4b61-a338-4f2901642899 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:57.093574 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:57.112689 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8cf86-d056-4fab-b0ef-2f1c7c4ed42a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:57.131478 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:57.154288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b1dd1-8440-4b61-a338-4f2901642899 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:57.154529 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843f991b-8627-4b8f-a0ba-15aef9e894fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:57.155215 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f991b-8627-4b8f-a0ba-15aef9e894fa None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:57.155595 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f991b-8627-4b8f-a0ba-15aef9e894fa None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:57.156116 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f991b-8627-4b8f-a0ba-15aef9e894fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:57.192376 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c798e2-0210-497c-892f-7861887eaa19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:57.193347 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c798e2-0210-497c-892f-7861887eaa19 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:57.193771 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c798e2-0210-497c-892f-7861887eaa19 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:57.194220 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c798e2-0210-497c-892f-7861887eaa19 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:57.195341 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c798e2-0210-497c-892f-7861887eaa19 None admin] RBAC: Adding query filter params (name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:57.195931 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c798e2-0210-497c-892f-7861887eaa19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:57.197595 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c798e2-0210-497c-892f-7861887eaa19 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:57.206908 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 277/559] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 21:17:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:57.207906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c798e2-0210-497c-892f-7861887eaa19 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:57.239360 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843f991b-8627-4b8f-a0ba-15aef9e894fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:57.241129 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843f991b-8627-4b8f-a0ba-15aef9e894fa None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:57.317499 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c798e2-0210-497c-892f-7861887eaa19 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:57.335864 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:57.355385 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 283/560] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:17:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:17:57.356557 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843f991b-8627-4b8f-a0ba-15aef9e894fa None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:57.357732 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b3ba6-7963-407e-a610-4340953b2592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:57.358753 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b3ba6-7963-407e-a610-4340953b2592 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:57.359291 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b3ba6-7963-407e-a610-4340953b2592 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:57.359939 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b3ba6-7963-407e-a610-4340953b2592 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:57.369803 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37b3ba6-7963-407e-a610-4340953b2592 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:57.371656 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37b3ba6-7963-407e-a610-4340953b2592 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:57.389111 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-e37b3ba6-7963-407e-a610-4340953b2592 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 21:17:57.389111 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:57.389111 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:57.389111 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:57.389111 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:57.389111 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:57.389111 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:57.389111 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:57.389111 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:57.389111 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:57.389111 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:57.389111 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:57.389111 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:17:57.389111 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:17:57.389111 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:17:57.389111 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:17:57.390359 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:57.390359 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:57.390359 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:17:57.390359 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:17:57.390359 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:17:57.390359 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:17:57.390359 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:57.390359 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:57.390359 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:57.390359 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:57.390359 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:57.390359 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:57.390359 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:57.390359 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:57.390359 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:57.390359 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:57.390359 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:57.390359 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:57.391306 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:57.391306 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:57.391306 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:17:57.391306 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:17:57.391306 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:17:57.391306 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:17:57.391306 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:17:57.391306 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:17:57.391306 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 21:17:57.391306 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:57.393627 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 278/561] 10.222.0.65 () {56 vars in 1093 bytes} [Thu May 19 21:17:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:57.394764 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b3ba6-7963-407e-a610-4340953b2592 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:57.399416 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:57.481374 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843f991b-8627-4b8f-a0ba-15aef9e894fa None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:57.508282 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b3ba6-7963-407e-a610-4340953b2592 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:57.521646 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c643d57-d72f-45e6-90fd-4970a6f264f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:57.523849 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c643d57-d72f-45e6-90fd-4970a6f264f2 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:57.524324 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c643d57-d72f-45e6-90fd-4970a6f264f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:57.525051 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c643d57-d72f-45e6-90fd-4970a6f264f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:57.538044 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c643d57-d72f-45e6-90fd-4970a6f264f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:57.541293 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c643d57-d72f-45e6-90fd-4970a6f264f2 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:57.547067 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f2aa3-65ce-451d-8623-c20f889b06a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:57.547955 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f2aa3-65ce-451d-8623-c20f889b06a9 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:57.548306 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f2aa3-65ce-451d-8623-c20f889b06a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:57.548651 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f2aa3-65ce-451d-8623-c20f889b06a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:57.562854 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59f2aa3-65ce-451d-8623-c20f889b06a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:57.564610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59f2aa3-65ce-451d-8623-c20f889b06a9 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:57.567522 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-8c643d57-d72f-45e6-90fd-4970a6f264f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:57.567522 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:57.567522 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:57.567522 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:57.567522 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:57.567522 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:57.567522 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:57.567522 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:57.567522 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:57.567522 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:57.567522 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:57.567522 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:57.567522 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:17:57.567522 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:17:57.567522 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:17:57.569080 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:17:57.569080 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:57.569080 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:57.569080 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:57.569080 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:57.569080 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:57.569080 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:57.569080 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:57.569080 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:17:57.569080 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:57.569080 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:57.569080 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:57.569080 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:57.569080 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:57.569080 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:57.569080 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:57.569080 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:57.571821 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:17:57.571821 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:17:57.571821 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:57.571821 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:17:57.579579 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 284/562] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:17:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:57.580947 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c643d57-d72f-45e6-90fd-4970a6f264f2 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:57.587613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-f59f2aa3-65ce-451d-8623-c20f889b06a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:57.587613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:57.587613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:57.587613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:57.587613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:57.587613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:57.587613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:57.587613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:57.587613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:57.587613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:57.587613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:57.587613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:57.587613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:17:57.587613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:17:57.587613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:17:57.588590 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:17:57.588590 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:57.588590 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:57.588590 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:57.588590 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:57.588590 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:57.588590 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:57.588590 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:57.588590 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:57.588590 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:57.588590 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:57.588590 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:57.588590 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:57.588590 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:57.588590 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:57.588590 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:57.588590 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:57.589487 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:17:57.589487 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:17:57.589487 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:57.589487 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:57.589487 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 279/563] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:17:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:57.589487 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f2aa3-65ce-451d-8623-c20f889b06a9 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:57.628270 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:57.672380 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:57.706799 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c643d57-d72f-45e6-90fd-4970a6f264f2 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:57.742603 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f2aa3-65ce-451d-8623-c20f889b06a9 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:57.748803 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61a324-7785-4ad2-9878-e0746cf875dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:57.749498 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61a324-7785-4ad2-9878-e0746cf875dc None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:57.749784 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61a324-7785-4ad2-9878-e0746cf875dc None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:57.750061 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61a324-7785-4ad2-9878-e0746cf875dc None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:57.750970 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf61a324-7785-4ad2-9878-e0746cf875dc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:57.751290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf61a324-7785-4ad2-9878-e0746cf875dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:57.752782 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf61a324-7785-4ad2-9878-e0746cf875dc None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:57.784626 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 285/564] 10.222.0.65 () {56 vars in 1133 bytes} [Thu May 19 21:17:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:57.785611 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61a324-7785-4ad2-9878-e0746cf875dc None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:57.795284 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a3534-1d7d-4275-8715-55c4400a1d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:57.795969 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a3534-1d7d-4275-8715-55c4400a1d31 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:57.796251 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a3534-1d7d-4275-8715-55c4400a1d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:57.796524 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a3534-1d7d-4275-8715-55c4400a1d31 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:57.797449 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601a3534-1d7d-4275-8715-55c4400a1d31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:57.797756 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601a3534-1d7d-4275-8715-55c4400a1d31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:57.799398 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601a3534-1d7d-4275-8715-55c4400a1d31 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:57.814299 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 280/565] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:17:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:57.815162 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a3534-1d7d-4275-8715-55c4400a1d31 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:57.921794 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:57.930547 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61a324-7785-4ad2-9878-e0746cf875dc None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:57.943064 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a3534-1d7d-4275-8715-55c4400a1d31 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:57.963871 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:57.980504 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d072f3ab-bea7-4c9a-b01c-cf1ad49a7bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:57.981283 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072f3ab-bea7-4c9a-b01c-cf1ad49a7bec None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:57.981566 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072f3ab-bea7-4c9a-b01c-cf1ad49a7bec None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:57.981839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072f3ab-bea7-4c9a-b01c-cf1ad49a7bec None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:57.982709 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d072f3ab-bea7-4c9a-b01c-cf1ad49a7bec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:57.983440 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d072f3ab-bea7-4c9a-b01c-cf1ad49a7bec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:57.984385 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d072f3ab-bea7-4c9a-b01c-cf1ad49a7bec None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:57.990755 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e37123-c1c5-4468-9229-23fd89089ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:57.991988 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e37123-c1c5-4468-9229-23fd89089ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:57.993066 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e37123-c1c5-4468-9229-23fd89089ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:57.993357 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e37123-c1c5-4468-9229-23fd89089ca1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:57.999615 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 286/566] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:17:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:58.001053 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72241834-663a-43ed-9c1b-e66834b02cac None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:58.001380 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72241834-663a-43ed-9c1b-e66834b02cac None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:58.001657 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72241834-663a-43ed-9c1b-e66834b02cac None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:58.007110 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 287/567] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:58.008220 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72241834-663a-43ed-9c1b-e66834b02cac None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:58.009566 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e37123-c1c5-4468-9229-23fd89089ca1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:58.011216 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e37123-c1c5-4468-9229-23fd89089ca1 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:58.031478 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-62e37123-c1c5-4468-9229-23fd89089ca1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:58.031478 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:58.031478 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:58.031478 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:58.031478 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:58.031478 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:58.031478 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:58.031478 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:58.031478 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:58.031478 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:58.031478 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:58.031478 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:58.031478 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:17:58.031478 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:17:58.031478 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:17:58.032450 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:17:58.032450 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:58.032450 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:58.032450 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:58.032450 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:58.032450 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:58.032450 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:58.032450 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:58.032450 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:58.032450 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:58.032450 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:58.032450 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:58.032450 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:58.032450 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:58.032450 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:58.032450 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:58.032450 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:58.033348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:17:58.033348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:17:58.033348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:58.033348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:58.033348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 281/568] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:17:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:58.033633 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e37123-c1c5-4468-9229-23fd89089ca1 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:58.145998 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72241834-663a-43ed-9c1b-e66834b02cac None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:58.160795 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e37123-c1c5-4468-9229-23fd89089ca1 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:58.193763 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe7e5c-6cd1-48b6-8d3e-79e982f64f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:58.194641 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe7e5c-6cd1-48b6-8d3e-79e982f64f93 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:58.195120 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe7e5c-6cd1-48b6-8d3e-79e982f64f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:58.195589 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe7e5c-6cd1-48b6-8d3e-79e982f64f93 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:58.204178 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfe7e5c-6cd1-48b6-8d3e-79e982f64f93 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:58.205657 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfe7e5c-6cd1-48b6-8d3e-79e982f64f93 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:58.206613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b980d17-e5bd-4ff0-a4df-d7380567c5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:58.207430 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b980d17-e5bd-4ff0-a4df-d7380567c5f1 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:58.207713 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b980d17-e5bd-4ff0-a4df-d7380567c5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:58.207987 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b980d17-e5bd-4ff0-a4df-d7380567c5f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:58.218069 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b980d17-e5bd-4ff0-a4df-d7380567c5f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:58.219611 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b980d17-e5bd-4ff0-a4df-d7380567c5f1 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:58.220894 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-3bfe7e5c-6cd1-48b6-8d3e-79e982f64f93 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 21:17:58.220894 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:58.220894 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:58.220894 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:58.220894 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:58.220894 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:58.220894 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:58.220894 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:58.220894 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:58.220894 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:58.220894 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:58.220894 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:58.220894 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:17:58.220894 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:17:58.220894 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:17:58.220894 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:17:58.221885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:58.221885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:58.221885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:17:58.221885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:17:58.221885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:17:58.221885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:17:58.221885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:58.221885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:58.221885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:58.221885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:58.221885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:58.221885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:17:58.221885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:58.221885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:58.221885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:58.221885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:58.221885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:58.221885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:58.223124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:58.223124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:58.223124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:17:58.223124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:17:58.223124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:17:58.223124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:17:58.223124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:17:58.223124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:17:58.223124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 21:17:58.223124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:17:58.223124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 288/569] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:17:58 2022] GET /identity/v3/users/glance => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:58.223919 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c6d46-ea83-49fe-b06b-00f0e0dc3707 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:58.223919 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c6d46-ea83-49fe-b06b-00f0e0dc3707 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:58.223919 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c6d46-ea83-49fe-b06b-00f0e0dc3707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:58.239699 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-2b980d17-e5bd-4ff0-a4df-d7380567c5f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:17:58.239699 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:58.239699 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:58.239699 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:58.239699 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:58.239699 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:58.239699 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:58.239699 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:58.239699 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:58.239699 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:58.239699 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:58.239699 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:58.239699 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:17:58.239699 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:17:58.239699 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:17:58.240861 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:17:58.240861 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:58.240861 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:58.240861 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:58.240861 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:58.240861 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:58.240861 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:58.240861 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:58.240861 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:58.240861 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:58.240861 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:58.240861 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:58.240861 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:58.240861 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:58.240861 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:58.240861 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:58.240861 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:58.243712 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:17:58.243712 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:17:58.243712 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:17:58.243712 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:17:58.243712 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:17:58.243712 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:17:58.243712 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:17:58.243712 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:58.243712 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 282/570] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:17:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:58.243712 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b980d17-e5bd-4ff0-a4df-d7380567c5f1 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:58.244960 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b94c6d46-ea83-49fe-b06b-00f0e0dc3707 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:58.244960 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b94c6d46-ea83-49fe-b06b-00f0e0dc3707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:58.254804 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:58.272015 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:58.319551 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 289/571] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:17:58.321608 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fdb55-c6f9-4e53-bcfa-10f8ea6c8aa3 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:58.321608 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fdb55-c6f9-4e53-bcfa-10f8ea6c8aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:58.321608 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fdb55-c6f9-4e53-bcfa-10f8ea6c8aa3 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:58.322852 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 290/572] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:17:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:17:58.323590 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544fdb55-c6f9-4e53-bcfa-10f8ea6c8aa3 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:58.355747 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b980d17-e5bd-4ff0-a4df-d7380567c5f1 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:58.391058 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78bc5e6-9d99-4b97-9e68-18341e0c8fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:58.391767 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78bc5e6-9d99-4b97-9e68-18341e0c8fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:58.392047 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78bc5e6-9d99-4b97-9e68-18341e0c8fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:58.392299 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78bc5e6-9d99-4b97-9e68-18341e0c8fa6 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:58.393042 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78bc5e6-9d99-4b97-9e68-18341e0c8fa6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:58.393369 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78bc5e6-9d99-4b97-9e68-18341e0c8fa6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:58.394785 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78bc5e6-9d99-4b97-9e68-18341e0c8fa6 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:58.410836 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 283/573] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:17:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:58.411704 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78bc5e6-9d99-4b97-9e68-18341e0c8fa6 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:58.459259 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544fdb55-c6f9-4e53-bcfa-10f8ea6c8aa3 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:58.495450 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e990dc0a-07bb-4317-8ef8-ec53acd5b77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:58.496160 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e990dc0a-07bb-4317-8ef8-ec53acd5b77b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:58.496500 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e990dc0a-07bb-4317-8ef8-ec53acd5b77b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:58.496784 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e990dc0a-07bb-4317-8ef8-ec53acd5b77b None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:58.497645 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e990dc0a-07bb-4317-8ef8-ec53acd5b77b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:58.497954 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e990dc0a-07bb-4317-8ef8-ec53acd5b77b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:58.499428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e990dc0a-07bb-4317-8ef8-ec53acd5b77b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:58.510955 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:58.519685 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 291/574] 10.222.0.65 () {56 vars in 1137 bytes} [Thu May 19 21:17:58 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:58.521079 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a7f8f-489c-4a1e-b79f-3e71cb23e090 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:58.521372 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a7f8f-489c-4a1e-b79f-3e71cb23e090 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:58.521647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a7f8f-489c-4a1e-b79f-3e71cb23e090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:58.527533 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:58.532202 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78bc5e6-9d99-4b97-9e68-18341e0c8fa6 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:58.542471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-1b8a7f8f-489c-4a1e-b79f-3e71cb23e090 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:17:58.543686 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1b8a7f8f-489c-4a1e-b79f-3e71cb23e090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:17:58.576862 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c05cea5-7f47-4d74-a164-b29d1e6f4fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:58.577622 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c05cea5-7f47-4d74-a164-b29d1e6f4fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:58.577903 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c05cea5-7f47-4d74-a164-b29d1e6f4fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:58.578174 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c05cea5-7f47-4d74-a164-b29d1e6f4fc1 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:58.578994 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c05cea5-7f47-4d74-a164-b29d1e6f4fc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:58.579366 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c05cea5-7f47-4d74-a164-b29d1e6f4fc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:58.580811 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c05cea5-7f47-4d74-a164-b29d1e6f4fc1 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:58.593519 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 284/575] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 21:17:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:58.596191 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c05cea5-7f47-4d74-a164-b29d1e6f4fc1 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:58.625786 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 292/576] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:17:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:17:58.627181 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a7f8f-489c-4a1e-b79f-3e71cb23e090 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:58.719370 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c05cea5-7f47-4d74-a164-b29d1e6f4fc1 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:58.752440 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a7f8f-489c-4a1e-b79f-3e71cb23e090 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:58.770275 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b072f-c2f0-48f4-beda-3b8f7816d27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:58.772657 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b072f-c2f0-48f4-beda-3b8f7816d27b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:58.773107 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b072f-c2f0-48f4-beda-3b8f7816d27b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:58.773513 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b072f-c2f0-48f4-beda-3b8f7816d27b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:58.786038 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:58.786812 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73b072f-c2f0-48f4-beda-3b8f7816d27b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:58.792216 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef3602d-ff51-4bca-8878-2b0901b28ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:58.793028 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef3602d-ff51-4bca-8878-2b0901b28ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:58.793311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef3602d-ff51-4bca-8878-2b0901b28ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:58.793585 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef3602d-ff51-4bca-8878-2b0901b28ef6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:58.795310 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-b73b072f-c2f0-48f4-beda-3b8f7816d27b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:17:58.795310 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:58.795310 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:58.795310 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:58.795310 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:58.795310 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:58.795310 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:58.795310 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:58.795310 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:58.795310 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:58.795310 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:58.795310 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:58.795310 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:58.795310 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:58.795310 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:58.795310 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:58.796307 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:58.796307 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:58.796307 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:58.796307 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:58.796307 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:58.796307 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:58.796307 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:58.796307 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:58.796307 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:58.796307 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:58.796307 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:58.796307 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:58.796307 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:58.796307 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:58.796307 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:58.796307 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:58.797131 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:58.797131 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:58.797131 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:58.797131 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:58.797131 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:58.797131 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:58.797131 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:58.797131 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:58.797131 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:58.797131 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:58.797131 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:17:58.797131 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:58.801387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 285/577] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 21:17:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:58.802311 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b072f-c2f0-48f4-beda-3b8f7816d27b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:58.804761 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef3602d-ff51-4bca-8878-2b0901b28ef6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:58.805890 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef3602d-ff51-4bca-8878-2b0901b28ef6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:58.812785 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:58.822645 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-7ef3602d-ff51-4bca-8878-2b0901b28ef6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:17:58.822645 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:58.822645 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:58.822645 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:58.822645 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:58.822645 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:58.822645 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:58.822645 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:58.822645 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:58.822645 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:58.822645 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:58.822645 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:58.822645 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:17:58.822645 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:17:58.822645 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:17:58.826749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:17:58.826749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:58.826749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:58.826749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:58.826749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:58.826749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:58.826749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:58.826749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:58.826749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:17:58.826749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:58.826749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:58.826749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:58.826749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:58.826749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:58.826749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:58.826749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:58.826749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:58.830412 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:17:58.830412 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:17:58.830412 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:17:58.830412 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:17:58.830412 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:17:58.830412 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:17:58.830412 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:17:58.830412 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:17:58.830412 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 293/578] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:17:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:58.830412 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef3602d-ff51-4bca-8878-2b0901b28ef6 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:58.940399 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef3602d-ff51-4bca-8878-2b0901b28ef6 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:58.940656 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b072f-c2f0-48f4-beda-3b8f7816d27b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:58.981142 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4286c6-c91b-4f23-a4f2-526dc5d0c7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:58.981887 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4286c6-c91b-4f23-a4f2-526dc5d0c7bf None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:58.982172 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4286c6-c91b-4f23-a4f2-526dc5d0c7bf None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:58.982482 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4286c6-c91b-4f23-a4f2-526dc5d0c7bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:58.985231 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b26829-05fa-44f6-b5b3-23fc0ea14e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:58.985940 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b26829-05fa-44f6-b5b3-23fc0ea14e76 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:58.986157 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b26829-05fa-44f6-b5b3-23fc0ea14e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:58.986466 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b26829-05fa-44f6-b5b3-23fc0ea14e76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:58.987310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b26829-05fa-44f6-b5b3-23fc0ea14e76 None admin] RBAC: Adding query filter params (role.id=9dc26e65165b49d6916b54458f847755, scope.project.id=c4940befa9c4405699c72deedc8a2127, user.id=7e07984b36934aff9052541ef2a56f36) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:58.987619 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b26829-05fa-44f6-b5b3-23fc0ea14e76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:58.989067 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b26829-05fa-44f6-b5b3-23fc0ea14e76 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:58.998317 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 294/579] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:17:58 2022] GET /identity/v3/role_assignments?user.id=7e07984b36934aff9052541ef2a56f36&scope.project.id=c4940befa9c4405699c72deedc8a2127&role.id=9dc26e65165b49d6916b54458f847755 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:58.998317 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b26829-05fa-44f6-b5b3-23fc0ea14e76 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:59.003765 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4286c6-c91b-4f23-a4f2-526dc5d0c7bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:59.006916 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4286c6-c91b-4f23-a4f2-526dc5d0c7bf None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:59.038015 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-ba4286c6-c91b-4f23-a4f2-526dc5d0c7bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:59.038015 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:59.038015 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:59.038015 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:59.038015 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:59.038015 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:59.038015 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:59.038015 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:59.038015 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:59.038015 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:59.038015 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:59.038015 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:59.038015 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:17:59.038015 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:17:59.038015 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:17:59.039175 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:17:59.039175 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:59.039175 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:59.039175 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:59.039175 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:59.039175 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:59.039175 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:59.039175 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:59.039175 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:59.039175 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:59.039175 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:59.039175 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:59.039175 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:59.039175 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:59.039175 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:59.039175 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:59.039175 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:59.040070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:17:59.040070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:17:59.040070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:59.040070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:59.041329 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 286/580] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:17:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:59.043661 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4286c6-c91b-4f23-a4f2-526dc5d0c7bf None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:59.075715 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:59.122634 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:59.138770 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b26829-05fa-44f6-b5b3-23fc0ea14e76 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:59.183276 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4286c6-c91b-4f23-a4f2-526dc5d0c7bf None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:59.186192 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8eebb0-3886-4e5e-a404-9b1c75ddc27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:59.187502 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8eebb0-3886-4e5e-a404-9b1c75ddc27c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:59.187908 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8eebb0-3886-4e5e-a404-9b1c75ddc27c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:59.188290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8eebb0-3886-4e5e-a404-9b1c75ddc27c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:59.208154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8eebb0-3886-4e5e-a404-9b1c75ddc27c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:17:59.222914 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-cf8eebb0-3886-4e5e-a404-9b1c75ddc27c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:17:59.222914 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:59.222914 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:59.222914 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:59.222914 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:59.222914 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:59.222914 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:59.222914 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:59.222914 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:59.222914 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:59.222914 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:59.222914 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:59.222914 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:17:59.222914 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:17:59.222914 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:17:59.222914 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:17:59.223964 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:17:59.223964 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:17:59.223964 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:17:59.223964 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:17:59.223964 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:59.223964 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:59.223964 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:59.223964 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:59.223964 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:59.223964 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:59.223964 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:59.223964 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:17:59.223964 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:59.223964 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:59.223964 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:59.223964 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:59.224801 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:59.224801 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:59.224801 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:59.224801 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:59.224801 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:17:59.224801 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:17:59.224801 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:17:59.224801 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:17:59.224801 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:17:59.224801 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:17:59.224801 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:17:59.224801 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:17:59.226599 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 295/581] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 21:17:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:59.227763 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8eebb0-3886-4e5e-a404-9b1c75ddc27c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:59.250364 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98024d1d-8878-48ec-a8eb-f8d29706d73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:59.250927 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98024d1d-8878-48ec-a8eb-f8d29706d73a None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:59.251206 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98024d1d-8878-48ec-a8eb-f8d29706d73a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:59.251476 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98024d1d-8878-48ec-a8eb-f8d29706d73a None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:59.252237 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98024d1d-8878-48ec-a8eb-f8d29706d73a None admin] RBAC: Adding query filter params (name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:59.252539 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98024d1d-8878-48ec-a8eb-f8d29706d73a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:59.253958 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98024d1d-8878-48ec-a8eb-f8d29706d73a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:59.269775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 287/582] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 21:17:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:59.269775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98024d1d-8878-48ec-a8eb-f8d29706d73a None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:59.396569 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8eebb0-3886-4e5e-a404-9b1c75ddc27c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:59.424697 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:59.451597 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98024d1d-8878-48ec-a8eb-f8d29706d73a None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:59.456022 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c5de8-10fc-4431-b896-83cd03522226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:59.456955 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c5de8-10fc-4431-b896-83cd03522226 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:59.457428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c5de8-10fc-4431-b896-83cd03522226 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:59.457896 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c5de8-10fc-4431-b896-83cd03522226 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:59.459326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c5de8-10fc-4431-b896-83cd03522226 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:59.459855 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c5de8-10fc-4431-b896-83cd03522226 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:59.463347 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c5de8-10fc-4431-b896-83cd03522226 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:59.484263 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 296/583] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 21:17:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:59.485691 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c5de8-10fc-4431-b896-83cd03522226 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:59.493023 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:59.514353 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf03062a-1896-4ccf-af43-c74a7b1d041d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:59.515125 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf03062a-1896-4ccf-af43-c74a7b1d041d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:59.515580 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf03062a-1896-4ccf-af43-c74a7b1d041d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:59.516021 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf03062a-1896-4ccf-af43-c74a7b1d041d None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:59.516915 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf03062a-1896-4ccf-af43-c74a7b1d041d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:59.517326 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf03062a-1896-4ccf-af43-c74a7b1d041d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:59.519111 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf03062a-1896-4ccf-af43-c74a7b1d041d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:59.539570 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 288/584] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:17:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:59.541033 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf03062a-1896-4ccf-af43-c74a7b1d041d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:59.630667 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c5de8-10fc-4431-b896-83cd03522226 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:59.671101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5923845-7ca5-428a-8720-266357acecb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:59.671769 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5923845-7ca5-428a-8720-266357acecb9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:59.672050 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5923845-7ca5-428a-8720-266357acecb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:59.672338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5923845-7ca5-428a-8720-266357acecb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:59.673096 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5923845-7ca5-428a-8720-266357acecb9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:17:59.673401 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5923845-7ca5-428a-8720-266357acecb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:59.674845 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5923845-7ca5-428a-8720-266357acecb9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:59.685662 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 297/585] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 21:17:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:17:59.686566 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5923845-7ca5-428a-8720-266357acecb9 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:59.734752 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf03062a-1896-4ccf-af43-c74a7b1d041d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:59.763230 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:59.793113 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b370946-4b50-48e8-9a06-fc8b723212f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:59.793966 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b370946-4b50-48e8-9a06-fc8b723212f0 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:59.794460 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b370946-4b50-48e8-9a06-fc8b723212f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:59.794911 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b370946-4b50-48e8-9a06-fc8b723212f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:59.811540 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b370946-4b50-48e8-9a06-fc8b723212f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:59.813963 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b370946-4b50-48e8-9a06-fc8b723212f0 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:59.851657 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-2b370946-4b50-48e8-9a06-fc8b723212f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:59.851657 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:17:59.851657 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:17:59.851657 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:17:59.851657 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:17:59.851657 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:17:59.851657 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:17:59.851657 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:17:59.851657 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:17:59.851657 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:17:59.851657 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:17:59.851657 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:17:59.851657 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:17:59.851657 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:17:59.851657 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:17:59.852875 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:17:59.852875 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:17:59.852875 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:17:59.852875 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:17:59.852875 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:17:59.852875 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:17:59.852875 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:17:59.852875 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:17:59.852875 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:17:59.852875 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:17:59.852875 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:17:59.852875 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:17:59.852875 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:17:59.852875 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:17:59.852875 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:17:59.852875 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:17:59.852875 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:17:59.853952 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:17:59.853952 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:17:59.853952 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:17:59.853952 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:17:59.853952 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 289/586] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:17:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:17:59.862033 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b370946-4b50-48e8-9a06-fc8b723212f0 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:17:59.866977 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5923845-7ca5-428a-8720-266357acecb9 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:17:59.914886 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6142f-fe0f-454e-9137-c0e021bfd6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:17:59.916374 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6142f-fe0f-454e-9137-c0e021bfd6e9 None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:17:59.916816 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6142f-fe0f-454e-9137-c0e021bfd6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:17:59.917210 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6142f-fe0f-454e-9137-c0e021bfd6e9 None admin] PATH_INFO: `/v3/projects/c4940befa9c4405699c72deedc8a2127/users/e254671b21fe4661a381b9521810cf80/roles/9dc26e65165b49d6916b54458f847755` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:17:59.919410 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:17:59.941158 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e6142f-fe0f-454e-9137-c0e021bfd6e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4940befa9c4405699c72deedc8a2127, user_id=e254671b21fe4661a381b9521810cf80, role_id=9dc26e65165b49d6916b54458f847755)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:17:59.942675 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e6142f-fe0f-454e-9137-c0e021bfd6e9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:17:59.966431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 298/587] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 21:17:59 2022] PUT /identity/v3/projects/c4940befa9c4405699c72deedc8a2127/users/e254671b21fe4661a381b9521810cf80/roles/9dc26e65165b49d6916b54458f847755 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:17:59.968040 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6142f-fe0f-454e-9137-c0e021bfd6e9 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:00.086311 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b370946-4b50-48e8-9a06-fc8b723212f0 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:00.174825 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:00.175555 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a07dd2d-6116-420d-b755-2700e83f54ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:00.176733 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07dd2d-6116-420d-b755-2700e83f54ff None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:00.176928 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07dd2d-6116-420d-b755-2700e83f54ff None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:00.177293 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07dd2d-6116-420d-b755-2700e83f54ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:00.192454 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a07dd2d-6116-420d-b755-2700e83f54ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:00.196279 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a07dd2d-6116-420d-b755-2700e83f54ff None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:00.207898 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6142f-fe0f-454e-9137-c0e021bfd6e9 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:00.225106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-2a07dd2d-6116-420d-b755-2700e83f54ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:00.225106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:00.225106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:00.225106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:00.225106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:00.225106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:00.225106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:00.225106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:00.225106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:00.225106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:00.225106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:00.225106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:00.225106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:18:00.225106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:18:00.225106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:18:00.227036 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:18:00.227036 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:00.227036 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:00.227036 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:00.227036 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:00.227036 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:00.227036 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:00.227036 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:00.227036 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:00.227036 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:00.227036 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:00.227036 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:00.227036 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:00.227036 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:00.227036 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:00.227036 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:00.227036 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:00.229373 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:18:00.229373 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:18:00.229373 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:18:00.229373 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:18:00.229373 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:18:00.229373 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:18:00.229373 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:00.229373 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:00.229373 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 290/588] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:17:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:00.231238 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a07dd2d-6116-420d-b755-2700e83f54ff None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:00.287909 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c24b2e-aabe-4720-8305-5e25dde04d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:00.290135 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c24b2e-aabe-4720-8305-5e25dde04d2a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:00.290731 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c24b2e-aabe-4720-8305-5e25dde04d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:00.291226 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c24b2e-aabe-4720-8305-5e25dde04d2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:00.311145 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c24b2e-aabe-4720-8305-5e25dde04d2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:00.313908 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c24b2e-aabe-4720-8305-5e25dde04d2a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:00.348352 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-40c24b2e-aabe-4720-8305-5e25dde04d2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:00.348352 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:00.348352 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:00.348352 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:00.348352 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:00.348352 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:00.348352 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:00.348352 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:00.348352 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:00.348352 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:00.348352 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:00.348352 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:00.348352 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:00.348352 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:00.348352 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:00.350045 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:00.350045 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:00.350045 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:00.350045 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:00.350045 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:00.350045 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:00.350045 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:00.350045 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:00.350045 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:00.350045 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:00.350045 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:00.350045 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:00.350045 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:00.350045 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:00.350045 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:00.350045 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:00.350045 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:00.352031 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:00.352031 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:00.352031 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:00.352031 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:00.352031 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 299/589] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:17:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:00.352538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c24b2e-aabe-4720-8305-5e25dde04d2a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:00.421623 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:00.440356 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a07dd2d-6116-420d-b755-2700e83f54ff None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:00.507526 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9415ecd-e9e1-4312-80fd-bde522ff4cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:00.509060 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9415ecd-e9e1-4312-80fd-bde522ff4cbb None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:00.509783 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9415ecd-e9e1-4312-80fd-bde522ff4cbb None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:00.510599 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9415ecd-e9e1-4312-80fd-bde522ff4cbb None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:00.512074 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9415ecd-e9e1-4312-80fd-bde522ff4cbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:00.512902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9415ecd-e9e1-4312-80fd-bde522ff4cbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:00.515883 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9415ecd-e9e1-4312-80fd-bde522ff4cbb None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:00.546015 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 291/590] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:00.548162 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9415ecd-e9e1-4312-80fd-bde522ff4cbb None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:00.568824 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c24b2e-aabe-4720-8305-5e25dde04d2a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:00.640524 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f0f0c-101a-4390-a255-2f032a949d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:00.641766 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f0f0c-101a-4390-a255-2f032a949d41 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:00.642499 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f0f0c-101a-4390-a255-2f032a949d41 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:00.643075 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f0f0c-101a-4390-a255-2f032a949d41 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:00.644746 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f0f0c-101a-4390-a255-2f032a949d41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:00.645378 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f0f0c-101a-4390-a255-2f032a949d41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:00.648686 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f0f0c-101a-4390-a255-2f032a949d41 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:00.668720 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:00.678294 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 300/591] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 21:18:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:00.680399 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f0f0c-101a-4390-a255-2f032a949d41 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:00.681937 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9415ecd-e9e1-4312-80fd-bde522ff4cbb None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:00.718798 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5a12a-933d-426e-8ba5-f4c8ba4b9c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:00.719725 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5a12a-933d-426e-8ba5-f4c8ba4b9c2f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:00.719725 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5a12a-933d-426e-8ba5-f4c8ba4b9c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:00.720029 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5a12a-933d-426e-8ba5-f4c8ba4b9c2f None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:00.720702 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd5a12a-933d-426e-8ba5-f4c8ba4b9c2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:00.721004 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd5a12a-933d-426e-8ba5-f4c8ba4b9c2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:00.722410 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd5a12a-933d-426e-8ba5-f4c8ba4b9c2f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:00.734593 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 292/592] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:00.735432 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5a12a-933d-426e-8ba5-f4c8ba4b9c2f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:00.751365 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:00.841903 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5a12a-933d-426e-8ba5-f4c8ba4b9c2f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:00.890752 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b083f8e-7d4e-454a-9c6d-8257e20100f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:00.891562 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b083f8e-7d4e-454a-9c6d-8257e20100f7 None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:00.891973 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b083f8e-7d4e-454a-9c6d-8257e20100f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:00.892142 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b083f8e-7d4e-454a-9c6d-8257e20100f7 None admin] PATH_INFO: `/v3/projects/c4940befa9c4405699c72deedc8a2127/users/ea874ca4088d4e6c83c9d0497323cf6b/roles/9dc26e65165b49d6916b54458f847755` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:00.912379 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b083f8e-7d4e-454a-9c6d-8257e20100f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4940befa9c4405699c72deedc8a2127, user_id=ea874ca4088d4e6c83c9d0497323cf6b, role_id=9dc26e65165b49d6916b54458f847755)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:00.913999 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b083f8e-7d4e-454a-9c6d-8257e20100f7 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:00.917310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f0f0c-101a-4390-a255-2f032a949d41 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:00.937847 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 293/593] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 21:18:00 2022] PUT /identity/v3/projects/c4940befa9c4405699c72deedc8a2127/users/ea874ca4088d4e6c83c9d0497323cf6b/roles/9dc26e65165b49d6916b54458f847755 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:18:00.938478 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b083f8e-7d4e-454a-9c6d-8257e20100f7 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:01.005275 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b39d36-27cf-43bd-8d34-5bf3819729c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:01.006167 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b39d36-27cf-43bd-8d34-5bf3819729c8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:01.006481 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b39d36-27cf-43bd-8d34-5bf3819729c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:01.008534 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b39d36-27cf-43bd-8d34-5bf3819729c8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:01.026549 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:01.036465 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b39d36-27cf-43bd-8d34-5bf3819729c8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:01.042041 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b39d36-27cf-43bd-8d34-5bf3819729c8 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:01.078528 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-a9b39d36-27cf-43bd-8d34-5bf3819729c8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 21:18:01.078528 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:01.078528 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:01.078528 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:01.078528 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:01.078528 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:01.078528 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:01.078528 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:01.078528 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:01.078528 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:01.078528 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:01.078528 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:01.078528 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:18:01.078528 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:18:01.078528 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:18:01.078528 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:18:01.080326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:01.080326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:01.080326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:18:01.080326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:01.080326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:18:01.080326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:01.080326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:01.080326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:01.080326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:01.080326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:01.080326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:01.080326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:01.080326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:01.080326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:01.080326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:01.080326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:01.080326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:01.080326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:01.082154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:01.082154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:01.082154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:18:01.082154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:18:01.082154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:18:01.082154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:18:01.082154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:18:01.082154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:18:01.082154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 21:18:01.082154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:01.088654 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 301/594] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:18:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 409 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:01.095909 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b39d36-27cf-43bd-8d34-5bf3819729c8 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:01.157207 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b083f8e-7d4e-454a-9c6d-8257e20100f7 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:01.243111 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8760dd76-9249-4630-b034-1b7177ddd154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:01.246440 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760dd76-9249-4630-b034-1b7177ddd154 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:01.247386 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760dd76-9249-4630-b034-1b7177ddd154 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:01.248253 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760dd76-9249-4630-b034-1b7177ddd154 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:01.268058 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8760dd76-9249-4630-b034-1b7177ddd154 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:01.271144 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8760dd76-9249-4630-b034-1b7177ddd154 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:01.299993 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:01.308257 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-8760dd76-9249-4630-b034-1b7177ddd154 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 21:18:01.308257 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:01.308257 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:01.308257 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:01.308257 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:01.308257 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:01.308257 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:01.308257 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:01.308257 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:01.308257 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:01.308257 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:01.308257 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:01.308257 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:18:01.308257 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:18:01.308257 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:18:01.308257 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:18:01.309930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:01.309930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:01.309930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:18:01.309930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:01.309930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:18:01.309930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:01.309930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:01.309930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:01.309930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:01.309930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:01.309930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:01.309930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:01.309930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:01.309930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:01.309930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:01.309930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:01.309930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:01.309930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:01.313519 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:01.313519 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:01.313519 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:18:01.313519 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:18:01.313519 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:18:01.313519 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:18:01.313519 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:18:01.313519 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:18:01.313519 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 21:18:01.313519 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:01.318095 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 294/595] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 21:18:00 2022] GET /identity/v3/users/neutron => generated 85 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:01.328209 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8760dd76-9249-4630-b034-1b7177ddd154 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:01.362531 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b39d36-27cf-43bd-8d34-5bf3819729c8 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:01.423336 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048df61-b5b8-4065-aede-84ba6bff3dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:01.425963 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048df61-b5b8-4065-aede-84ba6bff3dfe None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:01.427050 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048df61-b5b8-4065-aede-84ba6bff3dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:01.427833 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048df61-b5b8-4065-aede-84ba6bff3dfe None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:01.429753 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c048df61-b5b8-4065-aede-84ba6bff3dfe None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:01.430624 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c048df61-b5b8-4065-aede-84ba6bff3dfe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:01.434580 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c048df61-b5b8-4065-aede-84ba6bff3dfe None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:01.477534 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 302/596] 10.222.0.65 () {56 vars in 1137 bytes} [Thu May 19 21:18:01 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:18:01.485588 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048df61-b5b8-4065-aede-84ba6bff3dfe None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:01.574294 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8760dd76-9249-4630-b034-1b7177ddd154 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:01.629164 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:01.655373 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff6c76c-aea4-414c-966b-7fb18a74b709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:01.657137 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6c76c-aea4-414c-966b-7fb18a74b709 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:01.657978 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6c76c-aea4-414c-966b-7fb18a74b709 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:01.659071 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6c76c-aea4-414c-966b-7fb18a74b709 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:01.661042 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff6c76c-aea4-414c-966b-7fb18a74b709 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:01.662187 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff6c76c-aea4-414c-966b-7fb18a74b709 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:01.666141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff6c76c-aea4-414c-966b-7fb18a74b709 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:01.708034 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 295/597] 10.222.0.65 () {56 vars in 1139 bytes} [Thu May 19 21:18:01 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:01.717586 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff6c76c-aea4-414c-966b-7fb18a74b709 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:01.800275 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048df61-b5b8-4065-aede-84ba6bff3dfe None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:01.843120 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:01.876437 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b4d80-784d-41f2-8b94-8ff3479982f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:01.878043 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b4d80-784d-41f2-8b94-8ff3479982f8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:01.878713 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b4d80-784d-41f2-8b94-8ff3479982f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:01.879327 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b4d80-784d-41f2-8b94-8ff3479982f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:01.901086 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48b4d80-784d-41f2-8b94-8ff3479982f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:01.904131 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48b4d80-784d-41f2-8b94-8ff3479982f8 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:01.938858 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-f48b4d80-784d-41f2-8b94-8ff3479982f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:01.938858 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:01.938858 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:01.938858 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:01.938858 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:01.938858 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:01.938858 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:01.938858 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:01.938858 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:01.938858 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:01.938858 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:01.938858 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:01.938858 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:01.938858 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:01.938858 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:01.939929 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:01.939929 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:01.939929 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:01.939929 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:01.939929 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:01.939929 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:01.939929 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:01.939929 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:01.939929 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:01.939929 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:01.939929 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:01.939929 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:01.939929 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:01.939929 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:01.939929 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:01.939929 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:01.939929 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:01.940932 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:01.940932 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:01.940932 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:01.940932 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:01.950352 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 303/598] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 459 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:18:01.953308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b4d80-784d-41f2-8b94-8ff3479982f8 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:02.003008 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff6c76c-aea4-414c-966b-7fb18a74b709 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:02.046036 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d4667-8aef-4d02-9a53-21d1d2b0ab48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:02.046946 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d4667-8aef-4d02-9a53-21d1d2b0ab48 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:02.047272 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d4667-8aef-4d02-9a53-21d1d2b0ab48 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:02.047581 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d4667-8aef-4d02-9a53-21d1d2b0ab48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:02.063923 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586d4667-8aef-4d02-9a53-21d1d2b0ab48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:02.066680 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586d4667-8aef-4d02-9a53-21d1d2b0ab48 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:02.092635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-586d4667-8aef-4d02-9a53-21d1d2b0ab48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:02.092635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:02.092635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:02.092635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:02.092635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:02.092635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:02.092635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:02.092635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:02.092635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:02.092635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:02.092635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:02.092635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:02.092635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:02.092635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:02.092635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:02.093606 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:02.093606 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:02.093606 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:02.093606 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:02.093606 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:02.093606 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:02.093606 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:02.093606 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:02.093606 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:02.093606 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:02.093606 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:02.093606 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:02.093606 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:02.093606 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:02.093606 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:02.093606 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:02.093606 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:02.094532 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:02.094532 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:02.094532 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:02.094532 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:02.094532 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 296/599] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:18:02.096590 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c65e23-b0da-4640-a225-98bab0da9ef2 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:02.096590 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c65e23-b0da-4640-a225-98bab0da9ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:02.096590 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c65e23-b0da-4640-a225-98bab0da9ef2 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:02.106328 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 297/600] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:18:02.107409 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c65e23-b0da-4640-a225-98bab0da9ef2 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:02.147787 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:02.150192 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b4d80-784d-41f2-8b94-8ff3479982f8 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:02.218122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5145532-3918-4693-b7b6-494eb3dec23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:02.219525 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5145532-3918-4693-b7b6-494eb3dec23b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:02.220156 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5145532-3918-4693-b7b6-494eb3dec23b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:02.220850 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5145532-3918-4693-b7b6-494eb3dec23b None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:02.222384 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5145532-3918-4693-b7b6-494eb3dec23b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:02.223080 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5145532-3918-4693-b7b6-494eb3dec23b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:02.225900 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5145532-3918-4693-b7b6-494eb3dec23b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:02.229267 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c65e23-b0da-4640-a225-98bab0da9ef2 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:02.256309 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 304/601] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:18:02.259001 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fdcba4-0bea-46d2-9f92-1ccacdf7bde3 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:02.259739 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fdcba4-0bea-46d2-9f92-1ccacdf7bde3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:02.260245 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fdcba4-0bea-46d2-9f92-1ccacdf7bde3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:02.271439 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55abaa35-5456-4ce5-847c-0d3039ab8004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:02.272929 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55abaa35-5456-4ce5-847c-0d3039ab8004 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:02.273659 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55abaa35-5456-4ce5-847c-0d3039ab8004 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:02.274160 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55abaa35-5456-4ce5-847c-0d3039ab8004 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:02.275141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55abaa35-5456-4ce5-847c-0d3039ab8004 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:02.275593 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55abaa35-5456-4ce5-847c-0d3039ab8004 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:02.277224 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55abaa35-5456-4ce5-847c-0d3039ab8004 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:02.291293 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-71fdcba4-0bea-46d2-9f92-1ccacdf7bde3 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:02.291795 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 298/602] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:02.292868 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55abaa35-5456-4ce5-847c-0d3039ab8004 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:02.293727 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-71fdcba4-0bea-46d2-9f92-1ccacdf7bde3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:02.339739 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:02.403409 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:02.414875 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55abaa35-5456-4ce5-847c-0d3039ab8004 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:02.421536 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 305/603] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:18:02.423318 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fdcba4-0bea-46d2-9f92-1ccacdf7bde3 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:02.458037 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a22de53-8d64-45f5-ae0d-795a7520bd8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:02.459193 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a22de53-8d64-45f5-ae0d-795a7520bd8a None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:02.459667 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a22de53-8d64-45f5-ae0d-795a7520bd8a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:02.460283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a22de53-8d64-45f5-ae0d-795a7520bd8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:02.468836 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a22de53-8d64-45f5-ae0d-795a7520bd8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:02.470703 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a22de53-8d64-45f5-ae0d-795a7520bd8a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:02.487177 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-0a22de53-8d64-45f5-ae0d-795a7520bd8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:02.487177 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:02.487177 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:02.487177 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:02.487177 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:02.487177 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:02.487177 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:02.487177 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:02.487177 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:02.487177 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:02.487177 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:02.487177 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:02.487177 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:18:02.487177 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:18:02.487177 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:18:02.488138 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:18:02.488138 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:02.488138 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:02.488138 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:02.488138 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:02.488138 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:02.488138 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:02.488138 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:02.488138 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:02.488138 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:02.488138 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:02.488138 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:02.488138 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:02.488138 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:02.488138 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:02.488138 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:02.488138 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:02.489016 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:18:02.489016 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:18:02.489016 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:18:02.489016 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:18:02.489016 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:18:02.489016 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:18:02.489016 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:02.489016 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:02.492692 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 299/604] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:18:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:02.493772 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a22de53-8d64-45f5-ae0d-795a7520bd8a None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:02.617230 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a22de53-8d64-45f5-ae0d-795a7520bd8a None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:02.622083 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fdcba4-0bea-46d2-9f92-1ccacdf7bde3 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:02.661602 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea773e86-9d62-4ae3-b463-2f4d8825907b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:02.662569 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea773e86-9d62-4ae3-b463-2f4d8825907b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:02.663025 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea773e86-9d62-4ae3-b463-2f4d8825907b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:02.663792 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea773e86-9d62-4ae3-b463-2f4d8825907b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:02.677703 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea773e86-9d62-4ae3-b463-2f4d8825907b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:18:02.687371 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-ea773e86-9d62-4ae3-b463-2f4d8825907b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:18:02.687371 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:02.687371 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:02.687371 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:02.687371 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:02.687371 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:02.687371 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:02.687371 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:02.687371 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:02.687371 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:02.687371 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:02.687371 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:02.687371 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:18:02.687371 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:18:02.687371 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:18:02.687371 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:18:02.688383 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:18:02.688383 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:18:02.688383 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:18:02.688383 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:18:02.688383 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:02.688383 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:02.688383 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:02.688383 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:02.688383 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:02.688383 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:02.688383 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:02.688383 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:02.688383 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:02.688383 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:02.688383 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:02.688383 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:02.689219 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:02.689219 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:02.689219 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:02.689219 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:02.689219 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:18:02.689219 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:18:02.689219 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:18:02.689219 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:18:02.689219 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:18:02.689219 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:18:02.689219 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:18:02.689219 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:02.689860 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab74df1-1cd6-4174-b79e-a59f5ea4e4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:02.689960 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab74df1-1cd6-4174-b79e-a59f5ea4e4d2 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:02.690586 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab74df1-1cd6-4174-b79e-a59f5ea4e4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:02.691996 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab74df1-1cd6-4174-b79e-a59f5ea4e4d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:02.693455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 300/605] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 21:18:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:02.694616 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea773e86-9d62-4ae3-b463-2f4d8825907b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:02.705097 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab74df1-1cd6-4174-b79e-a59f5ea4e4d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:02.707986 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab74df1-1cd6-4174-b79e-a59f5ea4e4d2 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:02.713791 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:02.736915 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-6ab74df1-1cd6-4174-b79e-a59f5ea4e4d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:02.736915 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:02.736915 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:02.736915 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:02.736915 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:02.736915 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:02.736915 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:02.736915 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:02.736915 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:02.736915 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:02.736915 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:02.736915 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:02.736915 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:18:02.736915 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:18:02.736915 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:18:02.738084 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:18:02.738084 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:02.738084 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:02.738084 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:02.738084 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:02.738084 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:02.738084 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:02.738084 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:02.738084 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:02.738084 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:02.738084 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:02.738084 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:02.738084 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:02.738084 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:02.738084 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:02.738084 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:02.738084 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:02.739019 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:18:02.739019 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:18:02.739019 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:18:02.739019 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:18:02.739019 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:18:02.739019 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:18:02.739019 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:02.739019 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:02.748073 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 306/606] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:18:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:02.748073 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab74df1-1cd6-4174-b79e-a59f5ea4e4d2 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:02.782700 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:02.854948 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea773e86-9d62-4ae3-b463-2f4d8825907b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:02.901221 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0881369-5cea-4d39-8159-8454e4cbf67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:02.902401 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0881369-5cea-4d39-8159-8454e4cbf67d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:02.902824 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0881369-5cea-4d39-8159-8454e4cbf67d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:02.903725 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0881369-5cea-4d39-8159-8454e4cbf67d None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:02.904998 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0881369-5cea-4d39-8159-8454e4cbf67d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:02.905449 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0881369-5cea-4d39-8159-8454e4cbf67d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:02.908003 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0881369-5cea-4d39-8159-8454e4cbf67d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:02.914051 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab74df1-1cd6-4174-b79e-a59f5ea4e4d2 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:02.934735 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 301/607] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 21:18:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:02.936093 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0881369-5cea-4d39-8159-8454e4cbf67d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:02.958118 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b08b986-d39b-4cce-a11c-b6a8b70fb30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:02.959074 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08b986-d39b-4cce-a11c-b6a8b70fb30f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:02.959536 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08b986-d39b-4cce-a11c-b6a8b70fb30f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:02.960082 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08b986-d39b-4cce-a11c-b6a8b70fb30f None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:02.961517 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b08b986-d39b-4cce-a11c-b6a8b70fb30f None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:02.962050 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b08b986-d39b-4cce-a11c-b6a8b70fb30f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:02.963648 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b08b986-d39b-4cce-a11c-b6a8b70fb30f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:02.978072 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 307/608] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 21:18:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:02.979552 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b08b986-d39b-4cce-a11c-b6a8b70fb30f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:03.024639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:03.061414 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0881369-5cea-4d39-8159-8454e4cbf67d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:03.085371 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:03.111116 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b08b986-d39b-4cce-a11c-b6a8b70fb30f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:03.117349 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdaa7a0-081b-4f76-9d33-7adc12de6d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:03.118525 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdaa7a0-081b-4f76-9d33-7adc12de6d3b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:03.118884 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdaa7a0-081b-4f76-9d33-7adc12de6d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:03.119270 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdaa7a0-081b-4f76-9d33-7adc12de6d3b None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:03.120775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdaa7a0-081b-4f76-9d33-7adc12de6d3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:03.121324 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdaa7a0-081b-4f76-9d33-7adc12de6d3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:03.123887 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdaa7a0-081b-4f76-9d33-7adc12de6d3b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:03.153451 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b3372-f998-46bc-aa3d-487fc1e8727a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:03.154410 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 302/609] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 21:18:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:03.154591 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b3372-f998-46bc-aa3d-487fc1e8727a None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:03.154833 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b3372-f998-46bc-aa3d-487fc1e8727a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:03.155109 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b3372-f998-46bc-aa3d-487fc1e8727a None admin] PATH_INFO: `/v3/projects/c4940befa9c4405699c72deedc8a2127/users/51852131d4e5448f8152a51159bd78e4/roles/9dc26e65165b49d6916b54458f847755` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:03.155779 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdaa7a0-081b-4f76-9d33-7adc12de6d3b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:03.172177 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828b3372-f998-46bc-aa3d-487fc1e8727a None admin] RBAC: Authorizing `identity:create_grant(project_id=c4940befa9c4405699c72deedc8a2127, user_id=51852131d4e5448f8152a51159bd78e4, role_id=9dc26e65165b49d6916b54458f847755)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:03.173674 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828b3372-f998-46bc-aa3d-487fc1e8727a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:03.191233 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 308/610] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 21:18:02 2022] PUT /identity/v3/projects/c4940befa9c4405699c72deedc8a2127/users/51852131d4e5448f8152a51159bd78e4/roles/9dc26e65165b49d6916b54458f847755 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:18:03.203809 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf0cd6-b9e7-4f4b-b839-e9daf6f8c8b2 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:03.204135 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf0cd6-b9e7-4f4b-b839-e9daf6f8c8b2 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:03.204450 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf0cd6-b9e7-4f4b-b839-e9daf6f8c8b2 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:03.210681 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 309/611] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:03 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:18:03.210681 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf0cd6-b9e7-4f4b-b839-e9daf6f8c8b2 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:03.289508 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdaa7a0-081b-4f76-9d33-7adc12de6d3b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:03.328544 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:03.337019 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f234dfa0-9a27-4ff8-bebc-99c724f241cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:03.338426 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234dfa0-9a27-4ff8-bebc-99c724f241cc None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:03.338921 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234dfa0-9a27-4ff8-bebc-99c724f241cc None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:03.339404 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234dfa0-9a27-4ff8-bebc-99c724f241cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:03.357594 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f234dfa0-9a27-4ff8-bebc-99c724f241cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:03.359525 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f234dfa0-9a27-4ff8-bebc-99c724f241cc None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:03.382773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-f234dfa0-9a27-4ff8-bebc-99c724f241cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:03.382773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:03.382773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:03.382773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:03.382773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:03.382773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:03.382773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:03.382773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:03.382773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:03.382773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:03.382773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:03.382773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:03.382773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:03.382773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:03.382773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:03.383744 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:03.383744 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:03.383744 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:03.383744 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:03.383744 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:03.383744 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:03.383744 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:03.383744 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:03.383744 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:03.383744 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:03.383744 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:03.383744 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:03.383744 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:03.383744 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:03.383744 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:03.383744 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:03.383744 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:03.384648 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:03.384648 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:03.384648 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:03.384648 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:03.385316 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 303/612] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:03.386594 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4bb3f2-2f5e-4856-9111-e12172e9a51e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:03.388699 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4bb3f2-2f5e-4856-9111-e12172e9a51e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:03.388699 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4bb3f2-2f5e-4856-9111-e12172e9a51e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:03.408253 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-cd4bb3f2-2f5e-4856-9111-e12172e9a51e None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:03.409488 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-cd4bb3f2-2f5e-4856-9111-e12172e9a51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:03.415388 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf0cd6-b9e7-4f4b-b839-e9daf6f8c8b2 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:03.465407 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:03.473252 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3044c803-4b2f-40e7-a75d-23a4e7dd906d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:03.474107 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3044c803-4b2f-40e7-a75d-23a4e7dd906d None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:03.476042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3044c803-4b2f-40e7-a75d-23a4e7dd906d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:03.476042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3044c803-4b2f-40e7-a75d-23a4e7dd906d None admin] PATH_INFO: `/v3/projects/c4940befa9c4405699c72deedc8a2127/users/2201e66caa4b4a57be09d845d7cb0023/roles/9dc26e65165b49d6916b54458f847755` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:03.499523 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3044c803-4b2f-40e7-a75d-23a4e7dd906d None admin] RBAC: Authorizing `identity:create_grant(project_id=c4940befa9c4405699c72deedc8a2127, user_id=2201e66caa4b4a57be09d845d7cb0023, role_id=9dc26e65165b49d6916b54458f847755)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:03.500982 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3044c803-4b2f-40e7-a75d-23a4e7dd906d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:03.509949 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 304/613] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:18:03.511048 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4bb3f2-2f5e-4856-9111-e12172e9a51e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:03.520236 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 310/614] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 21:18:03 2022] PUT /identity/v3/projects/c4940befa9c4405699c72deedc8a2127/users/2201e66caa4b4a57be09d845d7cb0023/roles/9dc26e65165b49d6916b54458f847755 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:18:03.535197 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3044c803-4b2f-40e7-a75d-23a4e7dd906d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:03.768692 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4bb3f2-2f5e-4856-9111-e12172e9a51e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:03.774624 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3044c803-4b2f-40e7-a75d-23a4e7dd906d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:03.787385 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:03.818937 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43461202-6e9e-4858-8873-be95ce656e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:03.819706 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43461202-6e9e-4858-8873-be95ce656e49 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:03.820169 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43461202-6e9e-4858-8873-be95ce656e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:03.820650 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43461202-6e9e-4858-8873-be95ce656e49 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:03.821410 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43461202-6e9e-4858-8873-be95ce656e49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:03.821720 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43461202-6e9e-4858-8873-be95ce656e49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:03.823453 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43461202-6e9e-4858-8873-be95ce656e49 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:03.835341 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b1233-844d-462e-8164-7ce07440aabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:03.836695 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b1233-844d-462e-8164-7ce07440aabe None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:03.837264 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b1233-844d-462e-8164-7ce07440aabe None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:03.837389 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 305/615] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:03.837864 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b1233-844d-462e-8164-7ce07440aabe None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:03.838889 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dee055-4923-40d6-9749-a7eb0f93811a None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:03.839138 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dee055-4923-40d6-9749-a7eb0f93811a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:03.839418 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dee055-4923-40d6-9749-a7eb0f93811a None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:03.840983 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 306/616] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:18:03.848440 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dee055-4923-40d6-9749-a7eb0f93811a None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:03.863532 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b1233-844d-462e-8164-7ce07440aabe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:18:03.875122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-a14b1233-844d-462e-8164-7ce07440aabe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:18:03.875122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:03.875122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:03.875122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:03.875122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:03.875122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:03.875122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:03.875122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:03.875122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:03.875122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:03.875122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:03.875122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:03.875122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:18:03.875122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:18:03.875122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:18:03.875122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:18:03.876147 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:18:03.876147 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:18:03.876147 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:18:03.876147 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:18:03.876147 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:03.876147 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:03.876147 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:03.876147 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:03.876147 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:03.876147 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:03.876147 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:03.876147 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:03.876147 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:03.876147 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:03.876147 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:03.876147 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:03.877218 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:03.877218 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:03.877218 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:03.877218 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:03.877218 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:18:03.877218 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:18:03.877218 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:18:03.877218 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:18:03.877218 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:18:03.877218 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:18:03.877218 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:18:03.877218 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:03.880872 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 311/617] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 21:18:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:18:03.882994 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509483b-b708-4b97-b6c6-b7abe6a71bb0 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:03.883500 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509483b-b708-4b97-b6c6-b7abe6a71bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:03.884142 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509483b-b708-4b97-b6c6-b7abe6a71bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:03.907821 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-4509483b-b708-4b97-b6c6-b7abe6a71bb0 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:03.910315 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4509483b-b708-4b97-b6c6-b7abe6a71bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:03.975192 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:03.986190 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dee055-4923-40d6-9749-a7eb0f93811a None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:04.022565 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 312/618] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:18:04.023912 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509483b-b708-4b97-b6c6-b7abe6a71bb0 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:04.045855 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc86524-fbc6-406d-b022-f9d4aa5f4956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:04.047021 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc86524-fbc6-406d-b022-f9d4aa5f4956 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:04.047485 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc86524-fbc6-406d-b022-f9d4aa5f4956 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:04.047965 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc86524-fbc6-406d-b022-f9d4aa5f4956 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:04.060492 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc86524-fbc6-406d-b022-f9d4aa5f4956 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:04.062314 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc86524-fbc6-406d-b022-f9d4aa5f4956 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:04.078593 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-abc86524-fbc6-406d-b022-f9d4aa5f4956 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 21:18:04.078593 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:04.078593 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:04.078593 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:04.078593 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:04.078593 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:04.078593 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:04.078593 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:04.078593 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:04.078593 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:04.078593 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:04.078593 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:04.078593 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:18:04.078593 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:18:04.078593 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:18:04.078593 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:18:04.079610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:04.079610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:04.079610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:18:04.079610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:04.079610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:18:04.079610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:04.079610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:04.079610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:04.079610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:04.079610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:04.079610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:04.079610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:04.079610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:04.079610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:04.079610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:04.079610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:04.079610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:04.079610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:04.080710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:04.080710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:04.080710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:18:04.080710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:18:04.080710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:18:04.080710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:18:04.080710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:18:04.080710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:18:04.080710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 21:18:04.080710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:04.083646 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 307/619] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 21:18:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:04.084662 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc86524-fbc6-406d-b022-f9d4aa5f4956 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:04.138505 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:04.187360 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509483b-b708-4b97-b6c6-b7abe6a71bb0 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:04.216917 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc86524-fbc6-406d-b022-f9d4aa5f4956 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:04.227696 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec4bf31-e5ca-4416-8e6e-aa7a31be183e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:04.228363 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4bf31-e5ca-4416-8e6e-aa7a31be183e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:04.228656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4bf31-e5ca-4416-8e6e-aa7a31be183e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:04.228931 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4bf31-e5ca-4416-8e6e-aa7a31be183e None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:04.229712 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec4bf31-e5ca-4416-8e6e-aa7a31be183e None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:04.230232 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec4bf31-e5ca-4416-8e6e-aa7a31be183e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:04.231629 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec4bf31-e5ca-4416-8e6e-aa7a31be183e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:04.253016 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 313/620] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 21:18:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:04.254705 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec4bf31-e5ca-4416-8e6e-aa7a31be183e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:04.256701 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd71e0a-9566-4e47-a577-421375042971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:04.257806 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd71e0a-9566-4e47-a577-421375042971 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:04.258334 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd71e0a-9566-4e47-a577-421375042971 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:04.258785 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd71e0a-9566-4e47-a577-421375042971 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:04.270606 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd71e0a-9566-4e47-a577-421375042971 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:18:04.279040 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-6fd71e0a-9566-4e47-a577-421375042971 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:18:04.279040 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:04.279040 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:04.279040 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:04.279040 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:04.279040 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:04.279040 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:04.279040 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:04.279040 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:04.279040 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:04.279040 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:04.279040 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:04.279040 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:18:04.279040 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:18:04.279040 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:18:04.279040 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:18:04.280031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:18:04.280031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:18:04.280031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:18:04.280031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:18:04.280031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:04.280031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:04.280031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:04.280031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:04.280031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:04.280031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:04.280031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:04.280031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:04.280031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:04.280031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:04.280031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:04.280031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:04.280878 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:04.280878 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:04.280878 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:04.280878 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:04.280878 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:18:04.280878 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:18:04.280878 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:18:04.280878 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:18:04.280878 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:18:04.280878 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:18:04.280878 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:18:04.280878 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:04.283756 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 308/621] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 21:18:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:04.285020 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd71e0a-9566-4e47-a577-421375042971 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:04.342764 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:04.382728 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec4bf31-e5ca-4416-8e6e-aa7a31be183e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:04.387275 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:04.395451 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd71e0a-9566-4e47-a577-421375042971 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:04.432766 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c357df-3b73-4fec-a7c1-a265d639e31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:04.433970 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c357df-3b73-4fec-a7c1-a265d639e31e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:04.434437 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c357df-3b73-4fec-a7c1-a265d639e31e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:04.434876 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c357df-3b73-4fec-a7c1-a265d639e31e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:04.435121 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89847826-982f-4a94-9368-ef6011b01a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:04.436362 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89847826-982f-4a94-9368-ef6011b01a39 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:04.436949 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89847826-982f-4a94-9368-ef6011b01a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:04.437533 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89847826-982f-4a94-9368-ef6011b01a39 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:04.439629 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89847826-982f-4a94-9368-ef6011b01a39 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:04.440229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89847826-982f-4a94-9368-ef6011b01a39 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:04.443131 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89847826-982f-4a94-9368-ef6011b01a39 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:04.449015 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c357df-3b73-4fec-a7c1-a265d639e31e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:04.450772 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c357df-3b73-4fec-a7c1-a265d639e31e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:04.470041 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-30c357df-3b73-4fec-a7c1-a265d639e31e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:04.470041 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:04.470041 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:04.470041 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:04.470041 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:04.470041 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:04.470041 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:04.470041 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:04.470041 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:04.470041 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:04.470041 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:04.470041 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:04.470041 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:04.470041 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:04.470041 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:04.471077 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:04.471077 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:04.471077 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:04.471077 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:04.471077 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:04.471077 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:04.471077 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:04.471077 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:04.471077 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:04.471077 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:04.471077 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:04.471077 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:04.471077 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:04.471077 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:04.471077 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:04.471077 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:04.471077 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:04.472037 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:04.472037 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:04.472037 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:04.472037 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:04.474010 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 309/623] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:04.475089 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c357df-3b73-4fec-a7c1-a265d639e31e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:04.476136 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 314/623] 10.222.0.65 () {56 vars in 1135 bytes} [Thu May 19 21:18:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:04.479530 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89847826-982f-4a94-9368-ef6011b01a39 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:04.602171 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c357df-3b73-4fec-a7c1-a265d639e31e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:04.604989 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:04.623290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89847826-982f-4a94-9368-ef6011b01a39 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:04.676402 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc0ac7-01f5-4fdc-919c-5ecbd5c4b112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:04.678214 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc0ac7-01f5-4fdc-919c-5ecbd5c4b112 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:04.679261 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc0ac7-01f5-4fdc-919c-5ecbd5c4b112 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:04.680045 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc0ac7-01f5-4fdc-919c-5ecbd5c4b112 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:04.681744 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dc0ac7-01f5-4fdc-919c-5ecbd5c4b112 None admin] RBAC: Adding query filter params (name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:04.682287 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dc0ac7-01f5-4fdc-919c-5ecbd5c4b112 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:04.684902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dc0ac7-01f5-4fdc-919c-5ecbd5c4b112 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:04.707800 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e02154-0747-40b3-abc5-52c4823e2e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:04.709122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e02154-0747-40b3-abc5-52c4823e2e4f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:04.709653 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e02154-0747-40b3-abc5-52c4823e2e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:04.710420 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e02154-0747-40b3-abc5-52c4823e2e4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:04.720545 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 310/624] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 21:18:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:04.720545 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc0ac7-01f5-4fdc-919c-5ecbd5c4b112 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:04.740316 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e02154-0747-40b3-abc5-52c4823e2e4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:04.744583 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e02154-0747-40b3-abc5-52c4823e2e4f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:04.747948 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:04.781107 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-53e02154-0747-40b3-abc5-52c4823e2e4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:04.781107 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:04.781107 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:04.781107 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:04.781107 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:04.781107 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:04.781107 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:04.781107 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:04.781107 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:04.781107 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:04.781107 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:04.781107 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:04.781107 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:04.781107 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:04.781107 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:04.782101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:04.782101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:04.782101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:04.782101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:04.782101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:04.782101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:04.782101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:04.782101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:04.782101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:04.782101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:04.782101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:04.782101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:04.782101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:04.782101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:04.782101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:04.782101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:04.782101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:04.783034 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:04.783034 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:04.783034 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:04.783034 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:04.791741 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 315/625] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:04.792956 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e02154-0747-40b3-abc5-52c4823e2e4f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:05.008530 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc0ac7-01f5-4fdc-919c-5ecbd5c4b112 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:05.111747 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e198d93-bca2-4024-9f7d-8e70dcedc3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:05.117094 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e198d93-bca2-4024-9f7d-8e70dcedc3da None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:05.117905 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e198d93-bca2-4024-9f7d-8e70dcedc3da None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:05.118617 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e198d93-bca2-4024-9f7d-8e70dcedc3da None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:05.119096 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e02154-0747-40b3-abc5-52c4823e2e4f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:05.123759 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e198d93-bca2-4024-9f7d-8e70dcedc3da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:05.124567 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e198d93-bca2-4024-9f7d-8e70dcedc3da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:05.127494 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e198d93-bca2-4024-9f7d-8e70dcedc3da None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:05.179278 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 311/626] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 460 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:05.183053 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e198d93-bca2-4024-9f7d-8e70dcedc3da None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:05.271918 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade01695-7c77-422e-a3f3-7d7137224354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:05.273393 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade01695-7c77-422e-a3f3-7d7137224354 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:05.274054 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade01695-7c77-422e-a3f3-7d7137224354 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:05.274576 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade01695-7c77-422e-a3f3-7d7137224354 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:05.331113 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade01695-7c77-422e-a3f3-7d7137224354 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:05.333852 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade01695-7c77-422e-a3f3-7d7137224354 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:05.343161 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:05.351219 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:05.419839 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e198d93-bca2-4024-9f7d-8e70dcedc3da None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:05.431790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-ade01695-7c77-422e-a3f3-7d7137224354 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:05.431790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:05.431790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:05.431790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:05.431790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:05.431790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:05.431790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:05.431790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:05.431790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:05.431790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:05.431790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:05.431790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:05.431790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:05.431790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:05.431790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:05.433413 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:05.433413 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:05.433413 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:05.433413 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:05.433413 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:05.433413 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:05.433413 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:05.433413 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:05.433413 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:05.433413 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:05.433413 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:05.433413 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:05.433413 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:05.433413 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:05.433413 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:05.433413 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:05.433413 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:05.435117 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:05.435117 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:05.435117 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:05.435117 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:05.446164 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 316/627] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 654 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:05.447853 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade01695-7c77-422e-a3f3-7d7137224354 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:05.503552 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4134e7c1-ea71-4e83-a3f3-f537cba56f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:05.505506 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134e7c1-ea71-4e83-a3f3-f537cba56f1f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:05.506387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134e7c1-ea71-4e83-a3f3-f537cba56f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:05.507267 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134e7c1-ea71-4e83-a3f3-f537cba56f1f None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:05.509049 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4134e7c1-ea71-4e83-a3f3-f537cba56f1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:05.510063 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4134e7c1-ea71-4e83-a3f3-f537cba56f1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:05.513402 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4134e7c1-ea71-4e83-a3f3-f537cba56f1f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:05.540396 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 312/628] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:05.542510 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4134e7c1-ea71-4e83-a3f3-f537cba56f1f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:05.661775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade01695-7c77-422e-a3f3-7d7137224354 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:05.724965 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5cf3f9-362c-49d2-9590-e764bfed42df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:05.726854 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cf3f9-362c-49d2-9590-e764bfed42df None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:05.727347 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cf3f9-362c-49d2-9590-e764bfed42df None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:05.727896 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cf3f9-362c-49d2-9590-e764bfed42df None admin] PATH_INFO: `/v3/users/nova` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:05.728483 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4134e7c1-ea71-4e83-a3f3-f537cba56f1f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:05.744802 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5cf3f9-362c-49d2-9590-e764bfed42df None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:05.748382 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5cf3f9-362c-49d2-9590-e764bfed42df None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:05.775050 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:05.784129 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-9e5cf3f9-362c-49d2-9590-e764bfed42df None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 21:18:05.784129 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:05.784129 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:05.784129 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:05.784129 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:05.784129 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:05.784129 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:05.784129 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:05.784129 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:05.784129 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:05.784129 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:05.784129 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:05.784129 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:18:05.784129 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:18:05.784129 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:18:05.784129 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:18:05.785308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:05.785308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:05.785308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:18:05.785308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:05.785308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:18:05.785308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:05.785308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:05.785308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:05.785308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:05.785308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:05.785308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:05.785308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:05.785308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:05.785308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:05.785308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:05.785308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:05.785308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:05.785308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:05.786272 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:05.786272 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:05.786272 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:18:05.786272 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:18:05.786272 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:18:05.786272 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:18:05.786272 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:18:05.786272 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:18:05.786272 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 21:18:05.786272 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:05.786810 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 317/629] 10.222.0.65 () {56 vars in 1093 bytes} [Thu May 19 21:18:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:05.788197 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5cf3f9-362c-49d2-9590-e764bfed42df None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:05.792164 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d44f4-d695-43bb-ade3-6d5390f983ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:05.793585 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d44f4-d695-43bb-ade3-6d5390f983ab None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:05.794066 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d44f4-d695-43bb-ade3-6d5390f983ab None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:05.794677 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d44f4-d695-43bb-ade3-6d5390f983ab None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:05.795548 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d44f4-d695-43bb-ade3-6d5390f983ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:05.795854 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d44f4-d695-43bb-ade3-6d5390f983ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:05.797272 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d44f4-d695-43bb-ade3-6d5390f983ab None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:05.817812 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 313/630] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:05.817812 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84505ea7-72e2-4070-88b8-bef22e473179 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:05.817812 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84505ea7-72e2-4070-88b8-bef22e473179 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:05.817812 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84505ea7-72e2-4070-88b8-bef22e473179 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:05.817812 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 314/631] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:18:05.817812 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84505ea7-72e2-4070-88b8-bef22e473179 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:05.895009 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:05.968924 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5cf3f9-362c-49d2-9590-e764bfed42df None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:05.984875 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84505ea7-72e2-4070-88b8-bef22e473179 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:06.013926 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460fb18-f8a0-487b-b7a4-23c5d594a24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:06.015218 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460fb18-f8a0-487b-b7a4-23c5d594a24c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:06.015651 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460fb18-f8a0-487b-b7a4-23c5d594a24c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:06.016259 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460fb18-f8a0-487b-b7a4-23c5d594a24c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:06.028342 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1460fb18-f8a0-487b-b7a4-23c5d594a24c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:06.030898 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1460fb18-f8a0-487b-b7a4-23c5d594a24c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:06.041748 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3342a92-188b-4c6c-85a3-9709576d62e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:06.042874 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3342a92-188b-4c6c-85a3-9709576d62e2 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:06.043258 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3342a92-188b-4c6c-85a3-9709576d62e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:06.043698 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3342a92-188b-4c6c-85a3-9709576d62e2 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:06.045119 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3342a92-188b-4c6c-85a3-9709576d62e2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:06.045618 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3342a92-188b-4c6c-85a3-9709576d62e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:06.048130 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3342a92-188b-4c6c-85a3-9709576d62e2 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:06.053573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-1460fb18-f8a0-487b-b7a4-23c5d594a24c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:06.053573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:06.053573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:06.053573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:06.053573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:06.053573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:06.053573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:06.053573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:06.053573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:06.053573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:06.053573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:06.053573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:06.053573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:18:06.053573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:18:06.053573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:18:06.055098 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:18:06.055098 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:06.055098 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:06.055098 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:06.055098 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:06.055098 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:06.055098 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:06.055098 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:06.055098 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:06.055098 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:06.055098 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:06.055098 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:06.055098 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:06.055098 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:06.055098 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:06.055098 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:06.055098 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:06.056087 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:18:06.056087 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:18:06.056087 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:18:06.056087 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:18:06.056087 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:18:06.056087 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:18:06.056087 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:06.056087 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:06.056747 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 318/632] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:18:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:06.057659 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460fb18-f8a0-487b-b7a4-23c5d594a24c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:06.077793 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 315/633] 10.222.0.65 () {56 vars in 1133 bytes} [Thu May 19 21:18:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:06.079756 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c636d-db25-47e6-8c27-2c1de880deb1 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:06.080212 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c636d-db25-47e6-8c27-2c1de880deb1 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:06.081009 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c636d-db25-47e6-8c27-2c1de880deb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:06.100391 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-1b0c636d-db25-47e6-8c27-2c1de880deb1 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:06.101942 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1b0c636d-db25-47e6-8c27-2c1de880deb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:06.157891 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:06.187088 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 316/634] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:18:06.188444 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1560db91-d32c-4d8d-bf89-b692e3aa98d0 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:06.188731 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1560db91-d32c-4d8d-bf89-b692e3aa98d0 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:06.189016 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1560db91-d32c-4d8d-bf89-b692e3aa98d0 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:06.194279 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 317/635] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:18:06.194279 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1560db91-d32c-4d8d-bf89-b692e3aa98d0 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:06.194515 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460fb18-f8a0-487b-b7a4-23c5d594a24c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:06.220126 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:06.243430 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d0e116-30aa-4c74-ada8-601dc62dea9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:06.244528 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d0e116-30aa-4c74-ada8-601dc62dea9d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:06.244984 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d0e116-30aa-4c74-ada8-601dc62dea9d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:06.245372 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d0e116-30aa-4c74-ada8-601dc62dea9d None admin] PATH_INFO: `/v3/users/glance` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:06.254815 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d0e116-30aa-4c74-ada8-601dc62dea9d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:06.257191 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d0e116-30aa-4c74-ada8-601dc62dea9d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:06.274187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-39d0e116-30aa-4c74-ada8-601dc62dea9d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 21:18:06.274187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:06.274187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:06.274187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:06.274187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:06.274187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:06.274187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:06.274187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:06.274187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:06.274187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:06.274187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:06.274187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:06.274187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:18:06.274187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:18:06.274187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:18:06.274187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:18:06.276312 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:06.276312 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:06.276312 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:18:06.276312 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:06.276312 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:18:06.276312 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:06.276312 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:06.276312 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:06.276312 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:06.276312 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:06.276312 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:06.276312 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:06.276312 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:06.276312 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:06.276312 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:06.276312 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:06.276312 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:06.276312 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:06.278148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:06.278148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:06.278148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:18:06.278148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:18:06.278148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:18:06.278148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:18:06.278148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:18:06.278148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:18:06.278148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 21:18:06.278148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:06.278148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 319/636] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:18:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:06.278148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d0e116-30aa-4c74-ada8-601dc62dea9d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:06.336093 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1560db91-d32c-4d8d-bf89-b692e3aa98d0 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:06.383888 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964dbc1d-4fe4-428b-9c8a-66b089098657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:06.384929 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964dbc1d-4fe4-428b-9c8a-66b089098657 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:06.385367 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964dbc1d-4fe4-428b-9c8a-66b089098657 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:06.385811 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964dbc1d-4fe4-428b-9c8a-66b089098657 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:06.387232 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964dbc1d-4fe4-428b-9c8a-66b089098657 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:06.387741 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964dbc1d-4fe4-428b-9c8a-66b089098657 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:06.390132 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964dbc1d-4fe4-428b-9c8a-66b089098657 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:06.408949 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 318/637] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 21:18:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:06.409940 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d0e116-30aa-4c74-ada8-601dc62dea9d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:06.411505 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c32f4-b245-455f-8591-111d2a2ff850 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:06.412046 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c32f4-b245-455f-8591-111d2a2ff850 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:06.412529 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c32f4-b245-455f-8591-111d2a2ff850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:06.434200 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-9a7c32f4-b245-455f-8591-111d2a2ff850 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:06.435558 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9a7c32f4-b245-455f-8591-111d2a2ff850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:06.454676 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2d8fd-d238-4e69-a5b0-2bd3fb6beeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:06.455361 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2d8fd-d238-4e69-a5b0-2bd3fb6beeb8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:06.455611 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2d8fd-d238-4e69-a5b0-2bd3fb6beeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:06.455881 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2d8fd-d238-4e69-a5b0-2bd3fb6beeb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:06.479360 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:06.515724 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:06.523535 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 319/638] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:18:06.525021 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c32f4-b245-455f-8591-111d2a2ff850 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:06.565828 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b2d8fd-d238-4e69-a5b0-2bd3fb6beeb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:06.568604 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b2d8fd-d238-4e69-a5b0-2bd3fb6beeb8 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:06.659722 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c32f4-b245-455f-8591-111d2a2ff850 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:06.680128 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 320/639] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:18:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:18:06.681265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2d8fd-d238-4e69-a5b0-2bd3fb6beeb8 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:06.703702 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd233fb-923b-4947-9422-a09f9416cfdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:06.704953 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd233fb-923b-4947-9422-a09f9416cfdc None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:06.705401 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd233fb-923b-4947-9422-a09f9416cfdc None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:06.705850 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd233fb-923b-4947-9422-a09f9416cfdc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:06.723440 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd233fb-923b-4947-9422-a09f9416cfdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:18:06.737274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-8dd233fb-923b-4947-9422-a09f9416cfdc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:18:06.737274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:06.737274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:06.737274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:06.737274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:06.737274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:06.737274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:06.737274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:06.737274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:06.737274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:06.737274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:06.737274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:06.737274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:18:06.737274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:18:06.737274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:18:06.737274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:18:06.739213 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:18:06.739213 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:18:06.739213 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:18:06.739213 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:18:06.739213 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:06.739213 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:06.739213 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:06.739213 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:06.739213 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:06.739213 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:06.739213 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:06.739213 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:06.739213 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:06.739213 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:06.739213 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:06.739213 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:06.740775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:06.740775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:06.740775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:06.740775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:06.740775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:18:06.740775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:18:06.740775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:18:06.740775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:18:06.740775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:18:06.740775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:18:06.740775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:18:06.740775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:06.740775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 320/640] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 21:18:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:06.740775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd233fb-923b-4947-9422-a09f9416cfdc None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:06.782623 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:06.795214 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:06.831181 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2d8fd-d238-4e69-a5b0-2bd3fb6beeb8 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:06.881754 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0654e985-edd4-48aa-b62b-2b0f68c1bf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:06.882848 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654e985-edd4-48aa-b62b-2b0f68c1bf1a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:06.883288 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654e985-edd4-48aa-b62b-2b0f68c1bf1a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:06.883728 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654e985-edd4-48aa-b62b-2b0f68c1bf1a None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:06.885169 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0654e985-edd4-48aa-b62b-2b0f68c1bf1a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:06.885675 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0654e985-edd4-48aa-b62b-2b0f68c1bf1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:06.887946 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0654e985-edd4-48aa-b62b-2b0f68c1bf1a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:06.903982 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd233fb-923b-4947-9422-a09f9416cfdc None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:06.921489 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 321/641] 10.222.0.65 () {56 vars in 1137 bytes} [Thu May 19 21:18:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:06.921489 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0654e985-edd4-48aa-b62b-2b0f68c1bf1a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:06.948817 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defaf10a-3818-47ee-8728-3057fbe09cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:06.950068 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defaf10a-3818-47ee-8728-3057fbe09cd7 None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:06.950543 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defaf10a-3818-47ee-8728-3057fbe09cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:06.950987 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defaf10a-3818-47ee-8728-3057fbe09cd7 None admin] PATH_INFO: `/v3/projects/c4940befa9c4405699c72deedc8a2127/users/7e07984b36934aff9052541ef2a56f36/roles/9dc26e65165b49d6916b54458f847755` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:06.969255 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defaf10a-3818-47ee-8728-3057fbe09cd7 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4940befa9c4405699c72deedc8a2127, user_id=7e07984b36934aff9052541ef2a56f36, role_id=9dc26e65165b49d6916b54458f847755)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:06.970924 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defaf10a-3818-47ee-8728-3057fbe09cd7 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:06.988659 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 321/642] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 21:18:06 2022] PUT /identity/v3/projects/c4940befa9c4405699c72deedc8a2127/users/7e07984b36934aff9052541ef2a56f36/roles/9dc26e65165b49d6916b54458f847755 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:18:06.989979 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defaf10a-3818-47ee-8728-3057fbe09cd7 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:07.074033 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0654e985-edd4-48aa-b62b-2b0f68c1bf1a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:07.118781 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6f4ea8-26dc-4953-aaab-4adaf6b4dc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:07.119950 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f4ea8-26dc-4953-aaab-4adaf6b4dc3c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:07.120269 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f4ea8-26dc-4953-aaab-4adaf6b4dc3c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:07.120583 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f4ea8-26dc-4953-aaab-4adaf6b4dc3c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:07.122998 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:07.128103 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:07.138611 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6f4ea8-26dc-4953-aaab-4adaf6b4dc3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:18:07.148642 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-9c6f4ea8-26dc-4953-aaab-4adaf6b4dc3c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:18:07.148642 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:07.148642 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:07.148642 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:07.148642 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:07.148642 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:07.148642 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:07.148642 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:07.148642 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:07.148642 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:07.148642 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:07.148642 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:07.148642 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:18:07.148642 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:18:07.148642 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:18:07.148642 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:18:07.149938 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:18:07.149938 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:18:07.149938 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:18:07.149938 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:18:07.149938 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:07.149938 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:07.149938 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:07.149938 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:07.149938 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:07.149938 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:07.149938 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:07.149938 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:07.149938 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:07.149938 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:07.149938 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:07.149938 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:07.151603 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:07.151603 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:07.151603 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:07.151603 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:07.151603 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:18:07.151603 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:18:07.151603 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:18:07.151603 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:18:07.151603 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:18:07.151603 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:18:07.151603 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:18:07.151603 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:07.154561 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 322/643] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 21:18:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:07.154561 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6f4ea8-26dc-4953-aaab-4adaf6b4dc3c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:07.171785 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defaf10a-3818-47ee-8728-3057fbe09cd7 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:07.233586 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db7787b-eedd-439b-803d-503343dbb6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:07.235675 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db7787b-eedd-439b-803d-503343dbb6ba None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:07.235675 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db7787b-eedd-439b-803d-503343dbb6ba None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:07.235901 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db7787b-eedd-439b-803d-503343dbb6ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:07.253027 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db7787b-eedd-439b-803d-503343dbb6ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:07.255638 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db7787b-eedd-439b-803d-503343dbb6ba None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:07.289680 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-8db7787b-eedd-439b-803d-503343dbb6ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:07.289680 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:07.289680 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:07.289680 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:07.289680 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:07.289680 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:07.289680 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:07.289680 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:07.289680 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:07.289680 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:07.289680 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:07.289680 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:07.289680 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:07.289680 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:07.289680 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:07.290783 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:07.290783 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:07.290783 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:07.290783 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:07.290783 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:07.290783 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:07.290783 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:07.290783 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:07.290783 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:07.290783 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:07.290783 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:07.290783 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:07.290783 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:07.290783 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:07.290783 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:07.290783 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:07.290783 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:07.291995 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:07.291995 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:07.291995 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:07.291995 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:07.292363 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 322/644] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:07.293689 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db7787b-eedd-439b-803d-503343dbb6ba None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:07.309932 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6f4ea8-26dc-4953-aaab-4adaf6b4dc3c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:07.351556 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b5625-7929-421c-8a47-d2bd4552827e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:07.352233 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b5625-7929-421c-8a47-d2bd4552827e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:07.352517 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b5625-7929-421c-8a47-d2bd4552827e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:07.352785 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b5625-7929-421c-8a47-d2bd4552827e None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:07.353545 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902b5625-7929-421c-8a47-d2bd4552827e None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:07.353856 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902b5625-7929-421c-8a47-d2bd4552827e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:07.355299 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902b5625-7929-421c-8a47-d2bd4552827e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:07.364785 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 323/645] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 21:18:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:07.365629 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b5625-7929-421c-8a47-d2bd4552827e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:07.440341 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:07.491552 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db7787b-eedd-439b-803d-503343dbb6ba None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:07.492781 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b5625-7929-421c-8a47-d2bd4552827e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:07.534407 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442b35c-be03-4e0b-8f22-d3ba0940d4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:07.535182 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442b35c-be03-4e0b-8f22-d3ba0940d4bc None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:07.535535 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442b35c-be03-4e0b-8f22-d3ba0940d4bc None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:07.535875 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442b35c-be03-4e0b-8f22-d3ba0940d4bc None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:07.536725 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6442b35c-be03-4e0b-8f22-d3ba0940d4bc None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:07.537104 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6442b35c-be03-4e0b-8f22-d3ba0940d4bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:07.538712 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6442b35c-be03-4e0b-8f22-d3ba0940d4bc None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:07.542906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:07.549449 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 324/646] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 21:18:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:07.550127 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442b35c-be03-4e0b-8f22-d3ba0940d4bc None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:07.566982 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d1717-4d8c-414e-adff-e966ae9b2088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:07.568288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d1717-4d8c-414e-adff-e966ae9b2088 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:07.568773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d1717-4d8c-414e-adff-e966ae9b2088 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:07.569300 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d1717-4d8c-414e-adff-e966ae9b2088 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:07.677959 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27d1717-4d8c-414e-adff-e966ae9b2088 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:07.679881 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27d1717-4d8c-414e-adff-e966ae9b2088 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:07.683380 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442b35c-be03-4e0b-8f22-d3ba0940d4bc None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:07.705909 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:07.730583 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c11dc5-fa82-49f7-9256-e55c976d853c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:07.731570 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c11dc5-fa82-49f7-9256-e55c976d853c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:07.732125 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c11dc5-fa82-49f7-9256-e55c976d853c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:07.732593 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c11dc5-fa82-49f7-9256-e55c976d853c None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:07.733538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c11dc5-fa82-49f7-9256-e55c976d853c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:07.734025 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c11dc5-fa82-49f7-9256-e55c976d853c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:07.735904 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c11dc5-fa82-49f7-9256-e55c976d853c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:07.759138 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 325/647] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:07.759138 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c11dc5-fa82-49f7-9256-e55c976d853c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:07.811209 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 323/648] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:18:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:18:07.811871 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d1717-4d8c-414e-adff-e966ae9b2088 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:07.860814 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:07.904146 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c11dc5-fa82-49f7-9256-e55c976d853c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:07.963858 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ebf16e-a743-4998-a244-064410275335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:07.965063 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ebf16e-a743-4998-a244-064410275335 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:07.965506 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ebf16e-a743-4998-a244-064410275335 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:07.965786 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d1717-4d8c-414e-adff-e966ae9b2088 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:07.965952 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ebf16e-a743-4998-a244-064410275335 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:07.985443 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ebf16e-a743-4998-a244-064410275335 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:07.987877 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ebf16e-a743-4998-a244-064410275335 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:08.019384 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-04ebf16e-a743-4998-a244-064410275335 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:08.019384 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:08.019384 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:08.019384 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:08.019384 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:08.019384 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:08.019384 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:08.019384 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:08.019384 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:08.019384 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:08.019384 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:08.019384 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:08.019384 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:08.019384 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:08.019384 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:08.020389 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:08.020389 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:08.020389 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:08.020389 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:08.020389 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:08.020389 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:08.020389 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:08.020389 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:08.020389 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:08.020389 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:08.020389 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:08.020389 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:08.020389 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:08.020389 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:08.020389 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:08.020389 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:08.020389 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:08.021278 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:08.021278 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:08.021278 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:08.021278 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:08.021278 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 326/649] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:08.022191 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ebf16e-a743-4998-a244-064410275335 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:08.029556 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c6c61-c345-4118-8565-3b5a66dfa065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:08.030954 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c6c61-c345-4118-8565-3b5a66dfa065 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:08.031406 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c6c61-c345-4118-8565-3b5a66dfa065 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:08.031844 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c6c61-c345-4118-8565-3b5a66dfa065 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:08.050575 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00c6c61-c345-4118-8565-3b5a66dfa065 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:08.053010 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00c6c61-c345-4118-8565-3b5a66dfa065 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:08.071720 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:08.078476 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-b00c6c61-c345-4118-8565-3b5a66dfa065 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:08.078476 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:08.078476 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:08.078476 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:08.078476 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:08.078476 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:08.078476 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:08.078476 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:08.078476 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:08.078476 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:08.078476 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:08.078476 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:08.078476 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:08.078476 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:08.078476 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:08.079888 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:08.079888 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:08.079888 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:08.079888 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:08.079888 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:08.079888 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:08.079888 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:08.079888 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:08.079888 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:08.079888 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:08.079888 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:08.079888 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:08.079888 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:08.079888 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:08.079888 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:08.079888 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:08.079888 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:08.081567 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:08.081567 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:08.081567 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:08.081567 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:08.081567 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 324/650] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:08.081567 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c6c61-c345-4118-8565-3b5a66dfa065 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:08.165406 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ebf16e-a743-4998-a244-064410275335 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:08.194182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:08.207176 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66f259-354f-4cba-affa-628e5c1a08aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:08.208091 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66f259-354f-4cba-affa-628e5c1a08aa None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:08.208368 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66f259-354f-4cba-affa-628e5c1a08aa None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:08.208751 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66f259-354f-4cba-affa-628e5c1a08aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:08.216967 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db66f259-354f-4cba-affa-628e5c1a08aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:08.218364 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db66f259-354f-4cba-affa-628e5c1a08aa None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:08.220232 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c6c61-c345-4118-8565-3b5a66dfa065 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:08.234625 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-db66f259-354f-4cba-affa-628e5c1a08aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:08.234625 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:08.234625 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:08.234625 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:08.234625 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:08.234625 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:08.234625 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:08.234625 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:08.234625 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:08.234625 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:08.234625 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:08.234625 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:08.234625 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:18:08.234625 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:18:08.234625 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:18:08.241658 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:18:08.241658 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:08.241658 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:08.241658 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:08.241658 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:08.241658 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:08.241658 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:08.241658 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:08.241658 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:08.241658 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:08.241658 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:08.241658 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:08.241658 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:08.241658 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:08.241658 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:08.241658 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:08.241658 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:08.242749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:18:08.242749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:18:08.242749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:18:08.242749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:18:08.242749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:18:08.242749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:18:08.242749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:08.242749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:08.242749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 327/651] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:18:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:08.242749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66f259-354f-4cba-affa-628e5c1a08aa None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:08.277808 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a20327-7b10-444c-ba07-97ff16e41174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:08.279106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a20327-7b10-444c-ba07-97ff16e41174 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:08.279557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a20327-7b10-444c-ba07-97ff16e41174 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:08.280066 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a20327-7b10-444c-ba07-97ff16e41174 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:08.297515 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a20327-7b10-444c-ba07-97ff16e41174 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:08.299842 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a20327-7b10-444c-ba07-97ff16e41174 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:08.330429 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-66a20327-7b10-444c-ba07-97ff16e41174 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:08.330429 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:08.330429 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:08.330429 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:08.330429 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:08.330429 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:08.330429 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:08.330429 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:08.330429 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:08.330429 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:08.330429 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:08.330429 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:08.330429 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:08.330429 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:08.330429 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:08.332178 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:08.332178 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:08.332178 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:08.332178 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:08.332178 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:08.332178 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:08.332178 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:08.332178 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:08.332178 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:08.332178 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:08.332178 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:08.332178 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:08.332178 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:08.332178 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:08.332178 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:08.332178 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:08.332178 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:08.340124 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:08.340124 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:08.340124 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:08.340124 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:08.340124 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 325/652] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:08.340124 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a20327-7b10-444c-ba07-97ff16e41174 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:08.392201 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:08.407037 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66f259-354f-4cba-affa-628e5c1a08aa None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:08.451408 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5addc12d-6242-4510-bf10-c80f52d9a555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:08.452414 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5addc12d-6242-4510-bf10-c80f52d9a555 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:08.452754 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5addc12d-6242-4510-bf10-c80f52d9a555 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:08.453081 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5addc12d-6242-4510-bf10-c80f52d9a555 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:08.453909 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5addc12d-6242-4510-bf10-c80f52d9a555 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:08.454336 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5addc12d-6242-4510-bf10-c80f52d9a555 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:08.455993 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5addc12d-6242-4510-bf10-c80f52d9a555 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:08.469298 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a20327-7b10-444c-ba07-97ff16e41174 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:08.472052 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 328/653] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:08.473505 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5addc12d-6242-4510-bf10-c80f52d9a555 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:08.523069 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:08.531531 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af1788a-f646-48f6-9bbe-f8f757c12296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:08.532596 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1788a-f646-48f6-9bbe-f8f757c12296 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:08.533042 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1788a-f646-48f6-9bbe-f8f757c12296 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:08.533481 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1788a-f646-48f6-9bbe-f8f757c12296 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:08.534732 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af1788a-f646-48f6-9bbe-f8f757c12296 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:08.535232 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af1788a-f646-48f6-9bbe-f8f757c12296 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:08.537634 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af1788a-f646-48f6-9bbe-f8f757c12296 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:08.552988 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 326/654] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:08.554220 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af1788a-f646-48f6-9bbe-f8f757c12296 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:08.628938 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5addc12d-6242-4510-bf10-c80f52d9a555 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:08.675087 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad26950-dcd6-4127-a2e1-ac0c08de8f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:08.675760 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad26950-dcd6-4127-a2e1-ac0c08de8f7b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:08.676152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad26950-dcd6-4127-a2e1-ac0c08de8f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:08.676425 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad26950-dcd6-4127-a2e1-ac0c08de8f7b None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:08.677228 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad26950-dcd6-4127-a2e1-ac0c08de8f7b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:08.677533 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad26950-dcd6-4127-a2e1-ac0c08de8f7b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:08.679072 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad26950-dcd6-4127-a2e1-ac0c08de8f7b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:08.692430 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af1788a-f646-48f6-9bbe-f8f757c12296 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:08.698162 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 329/655] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 21:18:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:08.699024 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad26950-dcd6-4127-a2e1-ac0c08de8f7b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:08.715258 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:08.733646 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060915c-cf68-46e9-a7d4-e5ba9d6a4cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:08.734225 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060915c-cf68-46e9-a7d4-e5ba9d6a4cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:08.734530 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060915c-cf68-46e9-a7d4-e5ba9d6a4cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:08.734806 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060915c-cf68-46e9-a7d4-e5ba9d6a4cd0 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:08.735553 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d060915c-cf68-46e9-a7d4-e5ba9d6a4cd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:08.735861 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d060915c-cf68-46e9-a7d4-e5ba9d6a4cd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:08.737408 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d060915c-cf68-46e9-a7d4-e5ba9d6a4cd0 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:08.758209 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 327/656] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:08.758209 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060915c-cf68-46e9-a7d4-e5ba9d6a4cd0 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:08.845517 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:08.867759 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad26950-dcd6-4127-a2e1-ac0c08de8f7b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:08.906026 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060915c-cf68-46e9-a7d4-e5ba9d6a4cd0 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:08.912513 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5393c-09bd-45a2-90cf-848400ee93d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:08.913589 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5393c-09bd-45a2-90cf-848400ee93d3 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:08.914055 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5393c-09bd-45a2-90cf-848400ee93d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:08.914537 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5393c-09bd-45a2-90cf-848400ee93d3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:08.925350 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb5393c-09bd-45a2-90cf-848400ee93d3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:08.927425 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb5393c-09bd-45a2-90cf-848400ee93d3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:08.946027 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-ebb5393c-09bd-45a2-90cf-848400ee93d3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 21:18:08.946027 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:08.946027 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:08.946027 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:08.946027 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:08.946027 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:08.946027 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:08.946027 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:08.946027 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:08.946027 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:08.946027 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:08.946027 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:08.946027 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:18:08.946027 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:18:08.946027 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:18:08.946027 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:18:08.947898 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:08.947898 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:08.947898 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:18:08.947898 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:08.947898 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:18:08.947898 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:08.947898 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:08.947898 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:08.947898 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:08.947898 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:08.947898 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:08.947898 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:08.947898 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:08.947898 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:08.947898 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:08.947898 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:08.947898 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:08.947898 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:08.949750 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:08.949750 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:08.949750 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:18:08.949750 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:18:08.949750 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:18:08.949750 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:18:08.949750 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:18:08.949750 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:18:08.949750 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 21:18:08.949750 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:08.949750 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 330/657] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:18:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:08.952170 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5393c-09bd-45a2-90cf-848400ee93d3 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:08.955085 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fdfc85-dd73-4553-96f9-ff042ac9b306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:08.956881 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fdfc85-dd73-4553-96f9-ff042ac9b306 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:08.957141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fdfc85-dd73-4553-96f9-ff042ac9b306 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:08.957452 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fdfc85-dd73-4553-96f9-ff042ac9b306 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:08.967232 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fdfc85-dd73-4553-96f9-ff042ac9b306 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:08.969781 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fdfc85-dd73-4553-96f9-ff042ac9b306 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:08.988727 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-84fdfc85-dd73-4553-96f9-ff042ac9b306 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 21:18:08.988727 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:08.988727 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:08.988727 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:08.988727 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:08.988727 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:08.988727 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:08.988727 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:08.988727 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:08.988727 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:08.988727 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:08.988727 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:08.988727 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:18:08.988727 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:18:08.988727 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:18:08.988727 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:18:08.990977 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:08.990977 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:08.990977 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:18:08.990977 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:08.990977 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:18:08.990977 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:08.990977 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:08.990977 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:08.990977 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:08.990977 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:08.990977 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:08.990977 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:08.990977 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:08.990977 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:08.990977 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:08.990977 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:08.990977 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:08.990977 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:08.993106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:08.993106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:08.993106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:18:08.993106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:18:08.993106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:18:08.993106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:18:08.993106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:18:08.993106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:18:08.993106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 21:18:08.993106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:08.993106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 328/658] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 21:18:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:08.993106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fdfc85-dd73-4553-96f9-ff042ac9b306 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:09.051108 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:09.090338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5393c-09bd-45a2-90cf-848400ee93d3 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:09.121810 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fdfc85-dd73-4553-96f9-ff042ac9b306 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:09.136869 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d81cb8-2e03-443b-8320-4725cafd3a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:09.137550 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d81cb8-2e03-443b-8320-4725cafd3a60 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:09.137889 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d81cb8-2e03-443b-8320-4725cafd3a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:09.138150 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d81cb8-2e03-443b-8320-4725cafd3a60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:09.139017 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d81cb8-2e03-443b-8320-4725cafd3a60 None admin] RBAC: Adding query filter params (role.id=9dc26e65165b49d6916b54458f847755, scope.project.id=c4940befa9c4405699c72deedc8a2127, user.id=e254671b21fe4661a381b9521810cf80) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:09.139323 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d81cb8-2e03-443b-8320-4725cafd3a60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:09.140818 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d81cb8-2e03-443b-8320-4725cafd3a60 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:09.142683 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:09.158278 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 331/659] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:18:08 2022] GET /identity/v3/role_assignments?user.id=e254671b21fe4661a381b9521810cf80&scope.project.id=c4940befa9c4405699c72deedc8a2127&role.id=9dc26e65165b49d6916b54458f847755 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:09.158278 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d81cb8-2e03-443b-8320-4725cafd3a60 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:09.191987 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b8d247-62b4-424f-9c39-fdbc0d115675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:09.193519 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8d247-62b4-424f-9c39-fdbc0d115675 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:09.193984 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8d247-62b4-424f-9c39-fdbc0d115675 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:09.194564 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8d247-62b4-424f-9c39-fdbc0d115675 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:09.209854 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b8d247-62b4-424f-9c39-fdbc0d115675 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:09.213873 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b8d247-62b4-424f-9c39-fdbc0d115675 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:09.248677 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-c4b8d247-62b4-424f-9c39-fdbc0d115675 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:09.248677 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:09.248677 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:09.248677 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:09.248677 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:09.248677 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:09.248677 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:09.248677 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:09.248677 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:09.248677 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:09.248677 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:09.248677 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:09.248677 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:18:09.248677 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:18:09.248677 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:18:09.250547 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:18:09.250547 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:09.250547 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:09.250547 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:09.250547 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:09.250547 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:09.250547 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:09.250547 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:09.250547 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:09.250547 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:09.250547 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:09.250547 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:09.250547 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:09.250547 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:09.250547 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:09.250547 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:09.250547 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:09.252272 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:18:09.252272 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:18:09.252272 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:18:09.252272 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:18:09.252272 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:18:09.252272 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:18:09.252272 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:09.252272 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:09.258795 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 329/660] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:18:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:09.259490 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b8d247-62b4-424f-9c39-fdbc0d115675 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:09.322005 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d81cb8-2e03-443b-8320-4725cafd3a60 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:09.382913 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f9e7e6-2761-4045-b2ec-0970d9fb9663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:09.385869 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f9e7e6-2761-4045-b2ec-0970d9fb9663 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:09.386384 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f9e7e6-2761-4045-b2ec-0970d9fb9663 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:09.386862 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f9e7e6-2761-4045-b2ec-0970d9fb9663 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:09.388026 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f9e7e6-2761-4045-b2ec-0970d9fb9663 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:09.388477 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f9e7e6-2761-4045-b2ec-0970d9fb9663 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:09.390140 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f9e7e6-2761-4045-b2ec-0970d9fb9663 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:09.395790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:09.428874 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 332/661] 10.222.0.65 () {56 vars in 1137 bytes} [Thu May 19 21:18:09 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:09.430913 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f9e7e6-2761-4045-b2ec-0970d9fb9663 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:09.452213 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b8d247-62b4-424f-9c39-fdbc0d115675 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:09.502567 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19fced4-f612-4dd8-b504-a60f19e9fe6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:09.503867 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19fced4-f612-4dd8-b504-a60f19e9fe6e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:09.504070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19fced4-f612-4dd8-b504-a60f19e9fe6e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:09.504840 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19fced4-f612-4dd8-b504-a60f19e9fe6e None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:09.505352 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19fced4-f612-4dd8-b504-a60f19e9fe6e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:09.505666 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19fced4-f612-4dd8-b504-a60f19e9fe6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:09.507735 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19fced4-f612-4dd8-b504-a60f19e9fe6e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:09.531159 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 330/662] 10.222.0.65 () {56 vars in 1139 bytes} [Thu May 19 21:18:09 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:09.532102 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19fced4-f612-4dd8-b504-a60f19e9fe6e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:09.565294 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f9e7e6-2761-4045-b2ec-0970d9fb9663 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:09.576657 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:09.611836 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a0b804-1420-47a1-ab81-4c9204d79760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:09.612897 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0b804-1420-47a1-ab81-4c9204d79760 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:09.613351 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0b804-1420-47a1-ab81-4c9204d79760 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:09.613787 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0b804-1420-47a1-ab81-4c9204d79760 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:09.615355 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a0b804-1420-47a1-ab81-4c9204d79760 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:09.615355 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a0b804-1420-47a1-ab81-4c9204d79760 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:09.617880 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a0b804-1420-47a1-ab81-4c9204d79760 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:09.633339 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 333/663] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 21:18:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:09.634352 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a0b804-1420-47a1-ab81-4c9204d79760 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:09.687212 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19fced4-f612-4dd8-b504-a60f19e9fe6e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:09.704873 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:09.727678 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b42dee1-01f0-48f0-8e82-fdb783b79e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:09.728899 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b42dee1-01f0-48f0-8e82-fdb783b79e38 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:09.729484 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b42dee1-01f0-48f0-8e82-fdb783b79e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:09.729936 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b42dee1-01f0-48f0-8e82-fdb783b79e38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:09.762014 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a0b804-1420-47a1-ab81-4c9204d79760 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:09.802781 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173b250a-f56e-4f89-b3e9-f593ccb1878c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:09.803950 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b250a-f56e-4f89-b3e9-f593ccb1878c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:09.804468 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b250a-f56e-4f89-b3e9-f593ccb1878c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:09.804988 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b250a-f56e-4f89-b3e9-f593ccb1878c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:09.831484 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b42dee1-01f0-48f0-8e82-fdb783b79e38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:09.833210 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b42dee1-01f0-48f0-8e82-fdb783b79e38 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:09.871406 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:09.903789 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173b250a-f56e-4f89-b3e9-f593ccb1878c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:09.905585 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173b250a-f56e-4f89-b3e9-f593ccb1878c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:09.963545 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 331/664] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:18:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:18:09.964424 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b42dee1-01f0-48f0-8e82-fdb783b79e38 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:09.988794 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:10.048659 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 334/665] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:18:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:18:10.050113 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd40642-f6b6-4b9f-abf8-8629466d1a06 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:10.050517 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd40642-f6b6-4b9f-abf8-8629466d1a06 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:10.050867 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd40642-f6b6-4b9f-abf8-8629466d1a06 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:10.056667 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 335/666] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:18:10.056667 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd40642-f6b6-4b9f-abf8-8629466d1a06 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:10.083577 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b42dee1-01f0-48f0-8e82-fdb783b79e38 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:10.122995 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:10.133469 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2259783a-4890-44de-8e48-9d95ce9ac2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:10.134709 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2259783a-4890-44de-8e48-9d95ce9ac2b0 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:10.135433 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2259783a-4890-44de-8e48-9d95ce9ac2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:10.135921 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2259783a-4890-44de-8e48-9d95ce9ac2b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:10.137134 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2259783a-4890-44de-8e48-9d95ce9ac2b0 None admin] RBAC: Adding query filter params (role.id=9dc26e65165b49d6916b54458f847755, scope.project.id=c4940befa9c4405699c72deedc8a2127, user.id=ea874ca4088d4e6c83c9d0497323cf6b) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:10.137459 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2259783a-4890-44de-8e48-9d95ce9ac2b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:10.139046 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2259783a-4890-44de-8e48-9d95ce9ac2b0 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:10.152893 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 332/667] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:18:09 2022] GET /identity/v3/role_assignments?user.id=ea874ca4088d4e6c83c9d0497323cf6b&scope.project.id=c4940befa9c4405699c72deedc8a2127&role.id=9dc26e65165b49d6916b54458f847755 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:10.152893 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2259783a-4890-44de-8e48-9d95ce9ac2b0 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:10.193369 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd40642-f6b6-4b9f-abf8-8629466d1a06 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:10.246717 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12af02-5fa6-40ba-813c-afecf04a1660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:10.247663 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12af02-5fa6-40ba-813c-afecf04a1660 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:10.248020 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12af02-5fa6-40ba-813c-afecf04a1660 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:10.248364 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12af02-5fa6-40ba-813c-afecf04a1660 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:10.262329 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12af02-5fa6-40ba-813c-afecf04a1660 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:10.264017 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12af02-5fa6-40ba-813c-afecf04a1660 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:10.286417 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-ef12af02-5fa6-40ba-813c-afecf04a1660 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:10.286417 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:10.286417 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:10.286417 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:10.286417 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:10.286417 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:10.286417 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:10.286417 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:10.286417 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:10.286417 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:10.286417 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:10.286417 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:10.286417 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:10.286417 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:10.286417 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:10.287372 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:10.287372 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:10.287372 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:10.287372 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:10.287372 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:10.287372 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:10.287372 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:10.287372 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:10.287372 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:10.287372 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:10.287372 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:10.287372 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:10.287372 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:10.287372 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:10.287372 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:10.287372 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:10.287372 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:10.288562 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:10.288562 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:10.288562 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:10.288562 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:10.289073 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 336/668] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:10.290874 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590a041-1a65-4b82-8737-2576f14aae03 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:10.291270 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590a041-1a65-4b82-8737-2576f14aae03 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:10.291656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590a041-1a65-4b82-8737-2576f14aae03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:10.298202 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:10.312564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-2590a041-1a65-4b82-8737-2576f14aae03 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:10.314848 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2590a041-1a65-4b82-8737-2576f14aae03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:10.387009 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2259783a-4890-44de-8e48-9d95ce9ac2b0 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:10.403973 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 337/669] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:18:10.405208 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2590a041-1a65-4b82-8737-2576f14aae03 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:10.427012 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae0010-0815-4132-a501-8270d3d02148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:10.430160 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae0010-0815-4132-a501-8270d3d02148 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:10.430700 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae0010-0815-4132-a501-8270d3d02148 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:10.431117 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae0010-0815-4132-a501-8270d3d02148 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:10.451725 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ae0010-0815-4132-a501-8270d3d02148 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:10.453898 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ae0010-0815-4132-a501-8270d3d02148 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:10.473682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-85ae0010-0815-4132-a501-8270d3d02148 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:10.473682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:10.473682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:10.473682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:10.473682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:10.473682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:10.473682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:10.473682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:10.473682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:10.473682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:10.473682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:10.473682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:10.473682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:10.473682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:10.473682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:10.474658 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:10.474658 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:10.474658 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:10.474658 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:10.474658 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:10.474658 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:10.474658 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:10.474658 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:10.474658 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:10.474658 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:10.474658 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:10.474658 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:10.474658 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:10.474658 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:10.474658 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:10.474658 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:10.474658 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:10.475578 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:10.475578 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:10.475578 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:10.475578 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:10.475578 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 333/670] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:10.476292 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae0010-0815-4132-a501-8270d3d02148 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:10.558422 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:10.574952 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2590a041-1a65-4b82-8737-2576f14aae03 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:10.633586 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b749307-f0a0-40bd-b5b9-71e5333c408a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:10.634624 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b749307-f0a0-40bd-b5b9-71e5333c408a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:10.634624 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b749307-f0a0-40bd-b5b9-71e5333c408a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:10.634907 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b749307-f0a0-40bd-b5b9-71e5333c408a None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:10.635599 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b749307-f0a0-40bd-b5b9-71e5333c408a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:10.635903 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b749307-f0a0-40bd-b5b9-71e5333c408a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:10.637356 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b749307-f0a0-40bd-b5b9-71e5333c408a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:10.639129 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:10.658525 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 338/671] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:10.658525 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b749307-f0a0-40bd-b5b9-71e5333c408a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:10.677689 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae0010-0815-4132-a501-8270d3d02148 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:10.738034 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541fc38-56c7-471f-87e6-641df6a658e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:10.739909 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541fc38-56c7-471f-87e6-641df6a658e5 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:10.740723 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541fc38-56c7-471f-87e6-641df6a658e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:10.742061 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541fc38-56c7-471f-87e6-641df6a658e5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:10.757268 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3541fc38-56c7-471f-87e6-641df6a658e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:18:10.768153 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-3541fc38-56c7-471f-87e6-641df6a658e5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:18:10.768153 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:10.768153 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:10.768153 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:10.768153 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:10.768153 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:10.768153 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:10.768153 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:10.768153 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:10.768153 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:10.768153 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:10.768153 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:10.768153 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:18:10.768153 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:18:10.768153 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:18:10.768153 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:18:10.769910 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:18:10.769910 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:18:10.769910 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:18:10.769910 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:18:10.769910 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:10.769910 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:10.769910 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:10.769910 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:10.769910 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:10.769910 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:10.769910 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:10.769910 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:10.769910 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:10.769910 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:10.769910 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:10.769910 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:10.771507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:10.771507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:10.771507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:10.771507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:10.771507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:18:10.771507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:18:10.771507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:18:10.771507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:18:10.771507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:18:10.771507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:18:10.771507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:18:10.771507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:10.780411 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 334/672] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 21:18:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:10.781906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541fc38-56c7-471f-87e6-641df6a658e5 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:10.789405 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b749307-f0a0-40bd-b5b9-71e5333c408a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:10.839289 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ea30f-beda-4f08-9fb3-bc37e111d7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:10.840357 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ea30f-beda-4f08-9fb3-bc37e111d7c3 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:10.841155 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ea30f-beda-4f08-9fb3-bc37e111d7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:10.841351 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ea30f-beda-4f08-9fb3-bc37e111d7c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:10.858542 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309ea30f-beda-4f08-9fb3-bc37e111d7c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:10.859174 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309ea30f-beda-4f08-9fb3-bc37e111d7c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:10.860760 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309ea30f-beda-4f08-9fb3-bc37e111d7c3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:10.877456 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 339/673] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:10.878728 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ea30f-beda-4f08-9fb3-bc37e111d7c3 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:10.915817 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541fc38-56c7-471f-87e6-641df6a658e5 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:10.926751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:10.953129 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:10.984814 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df16bd56-6b3f-413e-b1ca-a559c1ee93b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:10.986434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16bd56-6b3f-413e-b1ca-a559c1ee93b4 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:10.986724 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16bd56-6b3f-413e-b1ca-a559c1ee93b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:10.987004 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16bd56-6b3f-413e-b1ca-a559c1ee93b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:10.998482 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df16bd56-6b3f-413e-b1ca-a559c1ee93b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:11.000387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df16bd56-6b3f-413e-b1ca-a559c1ee93b4 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:11.024238 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-df16bd56-6b3f-413e-b1ca-a559c1ee93b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:11.024238 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:11.024238 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:11.024238 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:11.024238 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:11.024238 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:11.024238 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:11.024238 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:11.024238 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:11.024238 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:11.024238 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:11.024238 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:11.024238 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:18:11.024238 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:18:11.024238 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:18:11.025326 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:18:11.025326 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:11.025326 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:11.025326 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:11.025326 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:11.025326 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:11.025326 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:11.025326 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:11.025326 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:11.025326 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:11.025326 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:11.025326 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:11.025326 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:11.025326 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:11.025326 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:11.025326 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:11.025326 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:11.028515 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:18:11.028515 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:18:11.028515 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:18:11.028515 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:18:11.028515 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:18:11.028515 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:18:11.028515 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:11.028515 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:11.030477 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 335/674] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:18:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:11.032002 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df16bd56-6b3f-413e-b1ca-a559c1ee93b4 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:11.034512 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ea30f-beda-4f08-9fb3-bc37e111d7c3 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:11.081771 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d0bb6-ea8e-4cbd-915e-7f88488b5d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:11.082521 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d0bb6-ea8e-4cbd-915e-7f88488b5d72 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:11.082842 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d0bb6-ea8e-4cbd-915e-7f88488b5d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:11.083080 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d0bb6-ea8e-4cbd-915e-7f88488b5d72 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:11.083852 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987d0bb6-ea8e-4cbd-915e-7f88488b5d72 None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:11.084160 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987d0bb6-ea8e-4cbd-915e-7f88488b5d72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:11.086487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987d0bb6-ea8e-4cbd-915e-7f88488b5d72 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:11.103292 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 340/675] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 21:18:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:11.104260 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d0bb6-ea8e-4cbd-915e-7f88488b5d72 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:11.236997 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d0bb6-ea8e-4cbd-915e-7f88488b5d72 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:11.259189 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:11.276188 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df16bd56-6b3f-413e-b1ca-a559c1ee93b4 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:11.283372 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3e28f9-e295-4e37-9577-23b1d33b9517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:11.283372 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e28f9-e295-4e37-9577-23b1d33b9517 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:11.283791 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e28f9-e295-4e37-9577-23b1d33b9517 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:11.283927 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e28f9-e295-4e37-9577-23b1d33b9517 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:11.284703 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3e28f9-e295-4e37-9577-23b1d33b9517 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:11.285016 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3e28f9-e295-4e37-9577-23b1d33b9517 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:11.286566 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3e28f9-e295-4e37-9577-23b1d33b9517 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:11.310714 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 341/676] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 21:18:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:11.313225 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3e28f9-e295-4e37-9577-23b1d33b9517 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:11.364632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4874d-75af-488b-8839-8a6f4dbd93bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:11.367156 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4874d-75af-488b-8839-8a6f4dbd93bb None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:11.368120 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4874d-75af-488b-8839-8a6f4dbd93bb None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:11.368904 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4874d-75af-488b-8839-8a6f4dbd93bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:11.383037 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea4874d-75af-488b-8839-8a6f4dbd93bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:11.386427 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea4874d-75af-488b-8839-8a6f4dbd93bb None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:11.424707 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-8ea4874d-75af-488b-8839-8a6f4dbd93bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:11.424707 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:11.424707 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:11.424707 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:11.424707 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:11.424707 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:11.424707 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:11.424707 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:11.424707 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:11.424707 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:11.424707 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:11.424707 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:11.424707 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:18:11.424707 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:18:11.424707 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:18:11.426507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:18:11.426507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:11.426507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:11.426507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:11.426507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:11.426507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:11.426507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:11.426507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:11.426507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:11.426507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:11.426507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:11.426507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:11.426507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:11.426507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:11.426507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:11.426507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:11.426507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:11.428191 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:18:11.428191 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:18:11.428191 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:18:11.428191 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:18:11.428191 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:18:11.428191 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:18:11.428191 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:11.428191 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:11.433257 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 336/677] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:18:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 402 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:11.435256 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4874d-75af-488b-8839-8a6f4dbd93bb None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:11.470934 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3e28f9-e295-4e37-9577-23b1d33b9517 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:11.517108 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:11.540223 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401fdadf-03f6-428a-8fd6-ec1c42b576cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:11.541679 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fdadf-03f6-428a-8fd6-ec1c42b576cc None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:11.542179 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fdadf-03f6-428a-8fd6-ec1c42b576cc None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:11.542709 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fdadf-03f6-428a-8fd6-ec1c42b576cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:11.560635 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401fdadf-03f6-428a-8fd6-ec1c42b576cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:11.562349 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401fdadf-03f6-428a-8fd6-ec1c42b576cc None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:11.584311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-401fdadf-03f6-428a-8fd6-ec1c42b576cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:11.584311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:11.584311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:11.584311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:11.584311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:11.584311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:11.584311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:11.584311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:11.584311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:11.584311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:11.584311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:11.584311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:11.584311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:11.584311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:11.584311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:11.586075 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:11.586075 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:11.586075 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:11.586075 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:11.586075 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:11.586075 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:11.586075 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:11.586075 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:11.586075 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:11.586075 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:11.586075 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:11.586075 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:11.586075 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:11.586075 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:11.586075 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:11.586075 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:11.586075 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:11.588144 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:11.588144 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:11.588144 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:11.588144 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:11.588144 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 342/678] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:11.592004 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401fdadf-03f6-428a-8fd6-ec1c42b576cc None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:11.635595 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:11.694860 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4874d-75af-488b-8839-8a6f4dbd93bb None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:11.752382 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401fdadf-03f6-428a-8fd6-ec1c42b576cc None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:11.770176 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1cf0df-7175-4ad2-896c-edf9373422a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:11.771853 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1cf0df-7175-4ad2-896c-edf9373422a2 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:11.772586 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1cf0df-7175-4ad2-896c-edf9373422a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:11.773250 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1cf0df-7175-4ad2-896c-edf9373422a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:11.775748 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1cf0df-7175-4ad2-896c-edf9373422a2 None admin] RBAC: Adding query filter params (role.id=9dc26e65165b49d6916b54458f847755, scope.project.id=c4940befa9c4405699c72deedc8a2127, user.id=51852131d4e5448f8152a51159bd78e4) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:11.776547 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1cf0df-7175-4ad2-896c-edf9373422a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:11.779830 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1cf0df-7175-4ad2-896c-edf9373422a2 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:11.798589 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 337/679] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:18:11 2022] GET /identity/v3/role_assignments?user.id=51852131d4e5448f8152a51159bd78e4&scope.project.id=c4940befa9c4405699c72deedc8a2127&role.id=9dc26e65165b49d6916b54458f847755 => generated 608 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:11.800843 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1cf0df-7175-4ad2-896c-edf9373422a2 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:11.804598 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d295e1b1-a446-4f82-8ee3-32b4f0a957d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:11.805429 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295e1b1-a446-4f82-8ee3-32b4f0a957d9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:11.805789 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295e1b1-a446-4f82-8ee3-32b4f0a957d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:11.806137 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295e1b1-a446-4f82-8ee3-32b4f0a957d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:11.807157 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d295e1b1-a446-4f82-8ee3-32b4f0a957d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:11.814543 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d295e1b1-a446-4f82-8ee3-32b4f0a957d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:11.814543 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d295e1b1-a446-4f82-8ee3-32b4f0a957d9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:11.831272 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 343/680] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 21:18:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:11.836655 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d295e1b1-a446-4f82-8ee3-32b4f0a957d9 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:12.025694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:12.042340 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:12.060133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1cf0df-7175-4ad2-896c-edf9373422a2 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:12.074516 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d295e1b1-a446-4f82-8ee3-32b4f0a957d9 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:12.138915 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442d60a-26c6-4eea-bcc8-c716379df90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:12.139635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442d60a-26c6-4eea-bcc8-c716379df90f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:12.139938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442d60a-26c6-4eea-bcc8-c716379df90f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:12.140202 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442d60a-26c6-4eea-bcc8-c716379df90f None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:12.140944 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1442d60a-26c6-4eea-bcc8-c716379df90f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:12.141251 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1442d60a-26c6-4eea-bcc8-c716379df90f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:12.142745 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1442d60a-26c6-4eea-bcc8-c716379df90f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:12.160648 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 338/681] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:12.167843 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442d60a-26c6-4eea-bcc8-c716379df90f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:12.187591 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd97b6-7321-4fef-92e1-80dd193ea9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:12.188304 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd97b6-7321-4fef-92e1-80dd193ea9d0 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:12.188586 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd97b6-7321-4fef-92e1-80dd193ea9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:12.188848 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd97b6-7321-4fef-92e1-80dd193ea9d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:12.189708 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd97b6-7321-4fef-92e1-80dd193ea9d0 None admin] RBAC: Adding query filter params (role.id=9dc26e65165b49d6916b54458f847755, scope.project.id=c4940befa9c4405699c72deedc8a2127, user.id=2201e66caa4b4a57be09d845d7cb0023) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:12.190022 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd97b6-7321-4fef-92e1-80dd193ea9d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:12.191582 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd97b6-7321-4fef-92e1-80dd193ea9d0 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:12.207635 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 344/682] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:18:11 2022] GET /identity/v3/role_assignments?user.id=2201e66caa4b4a57be09d845d7cb0023&scope.project.id=c4940befa9c4405699c72deedc8a2127&role.id=9dc26e65165b49d6916b54458f847755 => generated 608 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:12.221306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edccafa3-7750-45df-8fe6-5332c89125b2 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:12.221649 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edccafa3-7750-45df-8fe6-5332c89125b2 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:12.221870 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edccafa3-7750-45df-8fe6-5332c89125b2 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:12.223681 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 345/683] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:18:12.230434 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb73bf-7de6-447b-bf15-afa6cda39297 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:12.230726 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb73bf-7de6-447b-bf15-afa6cda39297 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:12.231006 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb73bf-7de6-447b-bf15-afa6cda39297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:12.252611 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c1eb73bf-7de6-447b-bf15-afa6cda39297 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:12.253874 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c1eb73bf-7de6-447b-bf15-afa6cda39297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:12.385584 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442d60a-26c6-4eea-bcc8-c716379df90f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:12.394475 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 346/684] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 21:18:12.421624 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb73bf-7de6-447b-bf15-afa6cda39297 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:12.456649 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371b1ea-1ac3-4005-a161-c1feaffc47ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:12.458449 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371b1ea-1ac3-4005-a161-c1feaffc47ca None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:12.459024 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371b1ea-1ac3-4005-a161-c1feaffc47ca None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:12.459675 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371b1ea-1ac3-4005-a161-c1feaffc47ca None admin] PATH_INFO: `/v3/users/swift` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:12.475143 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e371b1ea-1ac3-4005-a161-c1feaffc47ca None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:12.478524 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e371b1ea-1ac3-4005-a161-c1feaffc47ca None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:12.510414 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-e371b1ea-1ac3-4005-a161-c1feaffc47ca None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 21:18:12.510414 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:12.510414 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:12.510414 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:12.510414 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:12.510414 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:12.510414 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:12.510414 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:12.510414 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:12.510414 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:12.510414 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:12.510414 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:12.510414 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:18:12.510414 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:18:12.510414 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:18:12.510414 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:18:12.511557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:12.511557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:12.511557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:18:12.511557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:12.511557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:18:12.511557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:12.511557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:12.511557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:12.511557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:12.511557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:12.511557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:12.511557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:12.511557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:12.511557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:12.511557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:12.511557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:12.511557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:12.511557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:12.512637 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:12.512637 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:12.512637 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:18:12.512637 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:18:12.512637 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:18:12.512637 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:18:12.512637 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:18:12.512637 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:18:12.512637 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 21:18:12.512637 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:12.516034 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 339/685] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 21:18:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:12.525514 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371b1ea-1ac3-4005-a161-c1feaffc47ca None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:12.552268 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:12.620489 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:12.665863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb73bf-7de6-447b-bf15-afa6cda39297 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:12.747780 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13c682-f7a8-4092-8cfe-c4ec2d31b31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:12.749700 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13c682-f7a8-4092-8cfe-c4ec2d31b31f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:12.750487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13c682-f7a8-4092-8cfe-c4ec2d31b31f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:12.751185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13c682-f7a8-4092-8cfe-c4ec2d31b31f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:12.772843 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371b1ea-1ac3-4005-a161-c1feaffc47ca None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:12.791512 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd13c682-f7a8-4092-8cfe-c4ec2d31b31f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:18:12.809523 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-cd13c682-f7a8-4092-8cfe-c4ec2d31b31f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:18:12.809523 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:12.809523 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:12.809523 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:12.809523 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:12.809523 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:12.809523 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:12.809523 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:12.809523 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:12.809523 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:12.809523 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:12.809523 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:12.809523 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:18:12.809523 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:18:12.809523 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:18:12.809523 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:18:12.811341 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:18:12.811341 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:18:12.811341 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:18:12.811341 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:18:12.811341 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:12.811341 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:12.811341 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:12.811341 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:12.811341 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:12.811341 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:12.811341 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:12.811341 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:12.811341 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:12.811341 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:12.811341 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:12.811341 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:12.812907 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:12.812907 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:12.812907 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:12.812907 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:12.812907 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:18:12.812907 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:18:12.812907 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:18:12.812907 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:18:12.812907 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:18:12.812907 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:18:12.812907 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:18:12.812907 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:12.815250 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 347/686] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 21:18:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:12.821172 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13c682-f7a8-4092-8cfe-c4ec2d31b31f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:12.847476 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727e5de-3d76-48b0-893f-9ffee929bf0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:12.847476 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727e5de-3d76-48b0-893f-9ffee929bf0e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:12.847476 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727e5de-3d76-48b0-893f-9ffee929bf0e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:12.847476 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727e5de-3d76-48b0-893f-9ffee929bf0e None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:12.848294 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8727e5de-3d76-48b0-893f-9ffee929bf0e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:12.849020 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8727e5de-3d76-48b0-893f-9ffee929bf0e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:12.858377 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8727e5de-3d76-48b0-893f-9ffee929bf0e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:12.900472 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 340/687] 10.222.0.65 () {56 vars in 1135 bytes} [Thu May 19 21:18:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:12.909948 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727e5de-3d76-48b0-893f-9ffee929bf0e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:13.071855 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13c682-f7a8-4092-8cfe-c4ec2d31b31f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:13.149255 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d40795-d582-4d05-8cde-0f3569df5612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:13.150166 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:13.150377 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d40795-d582-4d05-8cde-0f3569df5612 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:13.150875 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d40795-d582-4d05-8cde-0f3569df5612 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:13.151755 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d40795-d582-4d05-8cde-0f3569df5612 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:13.152767 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d40795-d582-4d05-8cde-0f3569df5612 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:13.153315 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d40795-d582-4d05-8cde-0f3569df5612 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:13.155959 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d40795-d582-4d05-8cde-0f3569df5612 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:13.196803 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 348/688] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:18:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:18:13.203296 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d40795-d582-4d05-8cde-0f3569df5612 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:13.218601 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:13.295285 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727e5de-3d76-48b0-893f-9ffee929bf0e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:13.389893 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d8ea93-c45f-492a-88f3-3876bdc3c104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:13.391530 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d8ea93-c45f-492a-88f3-3876bdc3c104 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:13.392254 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d8ea93-c45f-492a-88f3-3876bdc3c104 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:13.393344 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d8ea93-c45f-492a-88f3-3876bdc3c104 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:13.522498 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d40795-d582-4d05-8cde-0f3569df5612 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:13.545157 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d8ea93-c45f-492a-88f3-3876bdc3c104 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:13.548837 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d8ea93-c45f-492a-88f3-3876bdc3c104 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:13.601674 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c3d05-172e-4152-b737-ae8217207a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:13.603707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c3d05-172e-4152-b737-ae8217207a9c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:13.604179 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c3d05-172e-4152-b737-ae8217207a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:13.604642 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c3d05-172e-4152-b737-ae8217207a9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:13.629770 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c3d05-172e-4152-b737-ae8217207a9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:13.633980 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c3d05-172e-4152-b737-ae8217207a9c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:13.676995 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-662c3d05-172e-4152-b737-ae8217207a9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:13.676995 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:13.676995 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:13.676995 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:13.676995 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:13.676995 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:13.676995 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:13.676995 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:13.676995 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:13.676995 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:13.676995 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:13.676995 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:13.676995 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:13.676995 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:13.676995 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:13.679467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:13.679467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:13.679467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:13.679467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:13.679467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:13.679467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:13.679467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:13.679467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:13.679467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:13.679467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:13.679467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:13.679467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:13.679467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:13.679467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:13.679467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:13.679467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:13.679467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:13.681124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:13.681124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:13.681124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:13.681124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:13.681124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 349/689] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 479 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:13.684997 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c9a2aa-d5f5-410f-87b2-7227d3b9be68 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:13.685334 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:13.685571 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c9a2aa-d5f5-410f-87b2-7227d3b9be68 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:13.685903 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c9a2aa-d5f5-410f-87b2-7227d3b9be68 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:13.693782 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 350/690] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:13 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:18:13.697822 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c9a2aa-d5f5-410f-87b2-7227d3b9be68 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:13.761492 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 341/691] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:18:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 852 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:18:13.764782 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ebfe8-66b9-4718-9611-e1eab4364c64 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:13.765550 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ebfe8-66b9-4718-9611-e1eab4364c64 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:13.766265 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ebfe8-66b9-4718-9611-e1eab4364c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:13.794718 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-0f4ebfe8-66b9-4718-9611-e1eab4364c64 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:13.797844 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0f4ebfe8-66b9-4718-9611-e1eab4364c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:13.823770 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:13.917451 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c9a2aa-d5f5-410f-87b2-7227d3b9be68 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:13.942788 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 342/692] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:18:13.944809 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4ebfe8-66b9-4718-9611-e1eab4364c64 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:14.005536 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d1af3f-70e5-4025-9247-acbc35f91e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:14.007160 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d1af3f-70e5-4025-9247-acbc35f91e28 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:14.007802 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d1af3f-70e5-4025-9247-acbc35f91e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:14.008163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d1af3f-70e5-4025-9247-acbc35f91e28 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:14.009744 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d1af3f-70e5-4025-9247-acbc35f91e28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:14.010307 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d1af3f-70e5-4025-9247-acbc35f91e28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:14.013185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d1af3f-70e5-4025-9247-acbc35f91e28 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:14.045278 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 351/693] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:14.046178 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d1af3f-70e5-4025-9247-acbc35f91e28 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:14.159737 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:14.171720 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4ebfe8-66b9-4718-9611-e1eab4364c64 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:14.204649 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d1af3f-70e5-4025-9247-acbc35f91e28 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:14.230611 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:14.248877 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294bc84a-ef6c-4c68-a0bc-caa901d7b7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:14.249695 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294bc84a-ef6c-4c68-a0bc-caa901d7b7ac None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:14.249980 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294bc84a-ef6c-4c68-a0bc-caa901d7b7ac None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:14.250271 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294bc84a-ef6c-4c68-a0bc-caa901d7b7ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:14.259183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6fa273-2e07-4646-b370-d1f0a7e4c5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:14.261407 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6fa273-2e07-4646-b370-d1f0a7e4c5ef None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:14.262365 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6fa273-2e07-4646-b370-d1f0a7e4c5ef None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:14.262841 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294bc84a-ef6c-4c68-a0bc-caa901d7b7ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:14.262952 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6fa273-2e07-4646-b370-d1f0a7e4c5ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:14.264314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294bc84a-ef6c-4c68-a0bc-caa901d7b7ac None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:14.285681 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-294bc84a-ef6c-4c68-a0bc-caa901d7b7ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:14.285681 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:14.285681 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:14.285681 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:14.285681 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:14.285681 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:14.285681 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:14.285681 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:14.285681 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:14.285681 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:14.285681 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:14.285681 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:14.285681 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:14.285681 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:14.285681 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:14.286707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:14.286707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:14.286707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:14.286707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:14.286707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:14.286707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:14.286707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:14.286707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:14.286707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:14.286707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:14.286707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:14.286707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:14.286707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:14.286707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:14.286707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:14.286707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:14.286707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:14.287594 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:14.287594 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:14.287594 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:14.287594 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:14.287848 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6fa273-2e07-4646-b370-d1f0a7e4c5ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:14.287923 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 352/694] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:14.288741 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294bc84a-ef6c-4c68-a0bc-caa901d7b7ac None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:14.289485 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6fa273-2e07-4646-b370-d1f0a7e4c5ef None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:14.313421 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-3d6fa273-2e07-4646-b370-d1f0a7e4c5ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:14.313421 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:14.313421 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:14.313421 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:14.313421 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:14.313421 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:14.313421 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:14.313421 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:14.313421 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:14.313421 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:14.313421 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:14.313421 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:14.313421 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:14.313421 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:14.313421 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:14.314428 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:14.314428 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:14.314428 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:14.314428 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:14.314428 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:14.314428 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:14.314428 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:14.314428 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:14.314428 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:14.314428 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:14.314428 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:14.314428 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:14.314428 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:14.314428 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:14.314428 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:14.314428 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:14.314428 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:14.315315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:14.315315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:14.315315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:14.315315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:14.322445 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 343/695] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:14.323745 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6fa273-2e07-4646-b370-d1f0a7e4c5ef None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:14.414088 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294bc84a-ef6c-4c68-a0bc-caa901d7b7ac None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:14.434725 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6fa273-2e07-4646-b370-d1f0a7e4c5ef None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:14.458545 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a122f5-81fc-4a88-8f9f-04745d1bbc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:14.459274 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a122f5-81fc-4a88-8f9f-04745d1bbc41 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:14.459684 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a122f5-81fc-4a88-8f9f-04745d1bbc41 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:14.460094 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a122f5-81fc-4a88-8f9f-04745d1bbc41 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:14.471047 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a122f5-81fc-4a88-8f9f-04745d1bbc41 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:14.472734 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a122f5-81fc-4a88-8f9f-04745d1bbc41 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:14.474471 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35082b7-e591-4bf2-a1fd-c6303cf4eb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:14.475588 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35082b7-e591-4bf2-a1fd-c6303cf4eb9d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:14.476336 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35082b7-e591-4bf2-a1fd-c6303cf4eb9d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:14.476819 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35082b7-e591-4bf2-a1fd-c6303cf4eb9d None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:14.477855 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35082b7-e591-4bf2-a1fd-c6303cf4eb9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:14.478410 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35082b7-e591-4bf2-a1fd-c6303cf4eb9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:14.480068 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35082b7-e591-4bf2-a1fd-c6303cf4eb9d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:14.494010 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 344/696] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:14.495328 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35082b7-e591-4bf2-a1fd-c6303cf4eb9d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:14.504242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-90a122f5-81fc-4a88-8f9f-04745d1bbc41 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 21:18:14.504242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:14.504242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:14.504242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:14.504242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:14.504242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:14.504242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:14.504242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:14.504242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:14.504242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:14.504242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:14.504242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:14.504242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:18:14.504242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:18:14.504242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:18:14.504242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:18:14.505267 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:14.505267 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:14.505267 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:18:14.505267 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:14.505267 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:18:14.505267 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:14.505267 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:14.505267 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:14.505267 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:14.505267 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:14.505267 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:14.505267 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:14.505267 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:14.505267 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:14.505267 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:14.505267 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:14.505267 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:14.505267 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:14.506325 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:14.506325 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:14.506325 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:18:14.506325 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:18:14.506325 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:18:14.506325 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:18:14.506325 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:18:14.506325 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:18:14.506325 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 21:18:14.506325 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:14.507201 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 353/697] 10.222.0.65 () {56 vars in 1093 bytes} [Thu May 19 21:18:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:14.509976 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a122f5-81fc-4a88-8f9f-04745d1bbc41 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:14.514459 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:14.573080 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:14.613913 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35082b7-e591-4bf2-a1fd-c6303cf4eb9d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:14.651981 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e425144-b0d6-4e21-874e-35e0d0e950bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:14.653039 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e425144-b0d6-4e21-874e-35e0d0e950bd None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:14.653539 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e425144-b0d6-4e21-874e-35e0d0e950bd None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:14.654008 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e425144-b0d6-4e21-874e-35e0d0e950bd None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:14.655011 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e425144-b0d6-4e21-874e-35e0d0e950bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:14.655551 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e425144-b0d6-4e21-874e-35e0d0e950bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:14.657193 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e425144-b0d6-4e21-874e-35e0d0e950bd None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:14.676782 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 345/698] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:14.678111 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e425144-b0d6-4e21-874e-35e0d0e950bd None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:14.723885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a122f5-81fc-4a88-8f9f-04745d1bbc41 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:14.757629 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:14.793804 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e425144-b0d6-4e21-874e-35e0d0e950bd None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:14.801212 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3c357e-2387-4f63-b38e-0d46ef31b678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:14.802416 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c357e-2387-4f63-b38e-0d46ef31b678 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:14.802907 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c357e-2387-4f63-b38e-0d46ef31b678 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:14.803438 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c357e-2387-4f63-b38e-0d46ef31b678 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:14.805056 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3c357e-2387-4f63-b38e-0d46ef31b678 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:14.808039 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3c357e-2387-4f63-b38e-0d46ef31b678 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:14.837243 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7624533-b074-4817-9dbd-4ef80acf17dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:14.838751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7624533-b074-4817-9dbd-4ef80acf17dc None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:14.839258 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7624533-b074-4817-9dbd-4ef80acf17dc None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:14.839864 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7624533-b074-4817-9dbd-4ef80acf17dc None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:14.841065 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7624533-b074-4817-9dbd-4ef80acf17dc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:14.841622 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7624533-b074-4817-9dbd-4ef80acf17dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:14.844031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7624533-b074-4817-9dbd-4ef80acf17dc None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:14.862945 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 354/699] 10.222.0.65 () {60 vars in 1132 bytes} [Thu May 19 21:18:14 2022] POST /identity/v3/users => generated 254 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:18:14.864661 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3c357e-2387-4f63-b38e-0d46ef31b678 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:14.871091 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 346/700] 10.222.0.65 () {56 vars in 1133 bytes} [Thu May 19 21:18:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:14.878380 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7624533-b074-4817-9dbd-4ef80acf17dc None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:15.036991 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:15.078344 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3c357e-2387-4f63-b38e-0d46ef31b678 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:15.142979 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7624533-b074-4817-9dbd-4ef80acf17dc None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:15.188698 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ece9e-ebd8-465e-9cdf-4f12cbe931bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:15.190027 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ece9e-ebd8-465e-9cdf-4f12cbe931bb None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:15.194451 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ece9e-ebd8-465e-9cdf-4f12cbe931bb None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:15.195024 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ece9e-ebd8-465e-9cdf-4f12cbe931bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:15.207046 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:15.210359 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10ece9e-ebd8-465e-9cdf-4f12cbe931bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:15.213338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10ece9e-ebd8-465e-9cdf-4f12cbe931bb None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:15.223580 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d66bc-9051-4a61-ab9c-0b5cbd82923c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:15.224750 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d66bc-9051-4a61-ab9c-0b5cbd82923c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:15.225314 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d66bc-9051-4a61-ab9c-0b5cbd82923c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:15.225793 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d66bc-9051-4a61-ab9c-0b5cbd82923c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:15.245403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-d10ece9e-ebd8-465e-9cdf-4f12cbe931bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:15.245403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:15.245403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:15.245403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:15.245403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:15.245403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:15.245403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:15.245403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:15.245403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:15.245403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:15.245403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:15.245403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:15.245403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:18:15.245403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:18:15.245403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:18:15.246425 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:18:15.246425 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:15.246425 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:15.246425 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:15.246425 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:15.246425 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:15.246425 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:15.246425 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:15.246425 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:15.246425 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:15.246425 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:15.246425 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:15.246425 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:15.246425 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:15.246425 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:15.246425 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:15.246425 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:15.247307 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:18:15.247307 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:18:15.247307 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:18:15.247307 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:18:15.247307 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:18:15.247307 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:18:15.247307 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:15.247307 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:15.248077 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 355/701] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:18:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:15.254415 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ece9e-ebd8-465e-9cdf-4f12cbe931bb None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:15.375251 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2d66bc-9051-4a61-ab9c-0b5cbd82923c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:15.379188 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2d66bc-9051-4a61-ab9c-0b5cbd82923c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:15.474480 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ece9e-ebd8-465e-9cdf-4f12cbe931bb None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:15.592632 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a038007-6d4b-4c5c-954c-e7f061470b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:15.593721 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a038007-6d4b-4c5c-954c-e7f061470b73 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:15.594261 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a038007-6d4b-4c5c-954c-e7f061470b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:15.594732 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a038007-6d4b-4c5c-954c-e7f061470b73 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:15.596001 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a038007-6d4b-4c5c-954c-e7f061470b73 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:15.596527 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a038007-6d4b-4c5c-954c-e7f061470b73 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:15.597253 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 347/702] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:18:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 721 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 21:18:15.600371 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a038007-6d4b-4c5c-954c-e7f061470b73 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:15.604988 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d66bc-9051-4a61-ab9c-0b5cbd82923c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:15.649797 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 356/703] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 21:18:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:15.656315 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a038007-6d4b-4c5c-954c-e7f061470b73 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:15.670188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:15.706962 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:15.798789 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d66bc-9051-4a61-ab9c-0b5cbd82923c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:15.846943 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a038007-6d4b-4c5c-954c-e7f061470b73 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:15.849259 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cca48a-614f-43a4-8ef9-66dcd150d149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:15.850458 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cca48a-614f-43a4-8ef9-66dcd150d149 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:15.851304 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cca48a-614f-43a4-8ef9-66dcd150d149 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:15.851596 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cca48a-614f-43a4-8ef9-66dcd150d149 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:15.868308 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cca48a-614f-43a4-8ef9-66dcd150d149 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:15.869943 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cca48a-614f-43a4-8ef9-66dcd150d149 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:15.891309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-c4cca48a-614f-43a4-8ef9-66dcd150d149 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:15.891309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:15.891309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:15.891309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:15.891309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:15.891309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:15.891309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:15.891309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:15.891309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:15.891309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:15.891309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:15.891309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:15.891309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:15.891309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:15.891309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:15.892348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:15.892348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:15.892348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:15.892348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:15.892348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:15.892348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:15.892348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:15.892348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:15.892348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:15.892348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:15.892348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:15.892348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:15.892348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:15.892348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:15.892348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:15.892348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:15.892348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:15.893246 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:15.893246 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:15.893246 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:15.893246 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:15.895806 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2ab26b-9de4-4ae2-b91e-ef233b8d9d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:15.896736 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ab26b-9de4-4ae2-b91e-ef233b8d9d4e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:15.897446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ab26b-9de4-4ae2-b91e-ef233b8d9d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:15.897890 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ab26b-9de4-4ae2-b91e-ef233b8d9d4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:15.898455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 348/704] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:15.899173 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2ab26b-9de4-4ae2-b91e-ef233b8d9d4e None admin] RBAC: Adding query filter params (role.id=9dc26e65165b49d6916b54458f847755, scope.project.id=c4940befa9c4405699c72deedc8a2127, user.id=7e07984b36934aff9052541ef2a56f36) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:15.899639 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2ab26b-9de4-4ae2-b91e-ef233b8d9d4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:15.900078 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5837507-e81a-4513-9a79-84ee505fc8c7 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:15.900308 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5837507-e81a-4513-9a79-84ee505fc8c7 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:15.900587 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5837507-e81a-4513-9a79-84ee505fc8c7 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:15.901904 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2ab26b-9de4-4ae2-b91e-ef233b8d9d4e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:15.906313 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 349/705] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:18:15.906313 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5837507-e81a-4513-9a79-84ee505fc8c7 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:15.920106 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 357/706] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:18:15 2022] GET /identity/v3/role_assignments?user.id=7e07984b36934aff9052541ef2a56f36&scope.project.id=c4940befa9c4405699c72deedc8a2127&role.id=9dc26e65165b49d6916b54458f847755 => generated 608 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:15.921929 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc1f7c-e662-4c37-905c-4aee9cd14b69 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:15.922524 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc1f7c-e662-4c37-905c-4aee9cd14b69 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:15.923033 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc1f7c-e662-4c37-905c-4aee9cd14b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:15.945639 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-ccbc1f7c-e662-4c37-905c-4aee9cd14b69 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:15.947968 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ccbc1f7c-e662-4c37-905c-4aee9cd14b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:16.049098 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:16.077092 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:16.078114 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5837507-e81a-4513-9a79-84ee505fc8c7 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:16.088640 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 358/707] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:18:16.090171 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d33e3-91f1-4731-af8a-868b296e88b6 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:16.090512 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d33e3-91f1-4731-af8a-868b296e88b6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:16.090790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d33e3-91f1-4731-af8a-868b296e88b6 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:16.092523 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 359/708] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:18:16.102665 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6edb7f8-2ef9-4310-8745-8e69b3274273 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:16.103002 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6edb7f8-2ef9-4310-8745-8e69b3274273 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:16.103217 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6edb7f8-2ef9-4310-8745-8e69b3274273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:16.130670 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c6edb7f8-2ef9-4310-8745-8e69b3274273 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:16.132849 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c6edb7f8-2ef9-4310-8745-8e69b3274273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:16.157120 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2636aaab-1ced-4232-a455-57910cd15c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:16.157738 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636aaab-1ced-4232-a455-57910cd15c1c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:16.158329 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636aaab-1ced-4232-a455-57910cd15c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:16.158623 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636aaab-1ced-4232-a455-57910cd15c1c None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:16.159383 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2636aaab-1ced-4232-a455-57910cd15c1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:16.159694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2636aaab-1ced-4232-a455-57910cd15c1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:16.161650 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2636aaab-1ced-4232-a455-57910cd15c1c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:16.180691 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 350/709] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:16.181628 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2636aaab-1ced-4232-a455-57910cd15c1c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:16.248299 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 360/710] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:18:16.250010 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6edb7f8-2ef9-4310-8745-8e69b3274273 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:16.329233 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2636aaab-1ced-4232-a455-57910cd15c1c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:16.387333 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6edb7f8-2ef9-4310-8745-8e69b3274273 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:16.406284 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf8fe67-8d17-435f-9e39-34ccad549c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:16.407804 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8fe67-8d17-435f-9e39-34ccad549c8c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:16.408259 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8fe67-8d17-435f-9e39-34ccad549c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:16.408774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8fe67-8d17-435f-9e39-34ccad549c8c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:16.410056 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf8fe67-8d17-435f-9e39-34ccad549c8c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:16.412821 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf8fe67-8d17-435f-9e39-34ccad549c8c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:16.422787 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:16.430829 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb4a9e-9168-4fe0-be05-66c4edbcc1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:16.431968 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb4a9e-9168-4fe0-be05-66c4edbcc1e3 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:16.432467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb4a9e-9168-4fe0-be05-66c4edbcc1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:16.432917 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb4a9e-9168-4fe0-be05-66c4edbcc1e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:16.440358 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-eaf8fe67-8d17-435f-9e39-34ccad549c8c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 21:18:16.440358 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:16.440358 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:18:16.440358 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:18:16.440358 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:18:16.440358 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:18:16.440358 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:18:16.440358 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:18:16.440358 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 21:18:16.440358 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:16.440358 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:18:16.440358 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:16.440358 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:16.440358 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:16.440358 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:16.440358 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:16.441490 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:16.441490 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:16.441490 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:16.441490 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:16.441490 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:16.441490 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:16.441490 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:16.441490 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:18:16.441490 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:18:16.441490 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:18:16.441490 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:18:16.441490 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:16.441490 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:16.441490 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:16.441490 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:16.441490 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:16.442390 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:16.442390 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:16.442390 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:16.442390 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:16.442390 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:16.442390 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:16.442390 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:16.442390 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:16.442390 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:16.442390 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:16.442390 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:16.442390 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:18:16.442390 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:18:16.442390 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 21:18:16.442390 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:16.443196 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 351/711] 10.222.0.65 () {56 vars in 1129 bytes} [Thu May 19 21:18:16 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:16.444416 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf8fe67-8d17-435f-9e39-34ccad549c8c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:16.445668 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb4a9e-9168-4fe0-be05-66c4edbcc1e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:16.447624 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb4a9e-9168-4fe0-be05-66c4edbcc1e3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:16.451446 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:16.477557 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-86cb4a9e-9168-4fe0-be05-66c4edbcc1e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:16.477557 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:16.477557 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:16.477557 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:16.477557 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:16.477557 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:16.477557 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:16.477557 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:16.477557 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:16.477557 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:16.477557 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:16.477557 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:16.477557 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:18:16.477557 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:18:16.477557 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:18:16.478628 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:18:16.478628 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:16.478628 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:16.478628 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:16.478628 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:16.478628 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:16.478628 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:16.478628 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:16.478628 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:16.478628 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:16.478628 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:16.478628 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:16.478628 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:16.478628 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:16.478628 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:16.478628 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:16.478628 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:16.479740 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:18:16.479740 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:18:16.479740 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:18:16.479740 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:18:16.479740 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:18:16.479740 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:18:16.479740 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:16.479740 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:16.482699 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 361/712] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:18:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:16.487743 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb4a9e-9168-4fe0-be05-66c4edbcc1e3 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:16.610055 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf8fe67-8d17-435f-9e39-34ccad549c8c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:16.639584 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb4a9e-9168-4fe0-be05-66c4edbcc1e3 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:16.680238 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21526e39-6a9d-47ac-8411-aadfc6a9aaaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:16.680840 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21526e39-6a9d-47ac-8411-aadfc6a9aaaf None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:16.681128 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21526e39-6a9d-47ac-8411-aadfc6a9aaaf None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:16.681414 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21526e39-6a9d-47ac-8411-aadfc6a9aaaf None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:16.681521 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9803e87f-958a-4025-ac64-ea4da56d33d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:16.682139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21526e39-6a9d-47ac-8411-aadfc6a9aaaf None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:16.682481 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21526e39-6a9d-47ac-8411-aadfc6a9aaaf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:16.682976 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9803e87f-958a-4025-ac64-ea4da56d33d8 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:16.683559 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9803e87f-958a-4025-ac64-ea4da56d33d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:16.683877 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21526e39-6a9d-47ac-8411-aadfc6a9aaaf None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:16.684064 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9803e87f-958a-4025-ac64-ea4da56d33d8 None admin] PATH_INFO: `/v3/services/network` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:16.685347 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9803e87f-958a-4025-ac64-ea4da56d33d8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:16.688081 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9803e87f-958a-4025-ac64-ea4da56d33d8 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:16.695040 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 362/713] 10.222.0.65 () {56 vars in 1124 bytes} [Thu May 19 21:18:16 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:16.695519 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21526e39-6a9d-47ac-8411-aadfc6a9aaaf None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:16.713288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-9803e87f-958a-4025-ac64-ea4da56d33d8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 21:18:16.713288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:16.713288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:18:16.713288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:18:16.713288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:18:16.713288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:18:16.713288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:18:16.713288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:18:16.713288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 21:18:16.713288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:16.713288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:18:16.713288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:16.713288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:16.713288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:16.713288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:16.713288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:16.713288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:16.714386 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:16.714386 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:16.714386 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:16.714386 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:16.714386 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:16.714386 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:16.714386 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:18:16.714386 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:18:16.714386 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:18:16.714386 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:18:16.714386 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:16.714386 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:16.714386 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:16.714386 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:16.714386 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:16.714386 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:16.715216 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:16.715216 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:16.715216 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:16.715216 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:16.715216 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:16.715216 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:16.715216 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:16.715216 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:16.715216 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:16.715216 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:16.715216 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:18:16.715216 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:18:16.715216 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 21:18:16.715216 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:16.722267 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 352/714] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:18:16 2022] GET /identity/v3/services/network => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:16.724030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9803e87f-958a-4025-ac64-ea4da56d33d8 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:16.759527 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:16.825956 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21526e39-6a9d-47ac-8411-aadfc6a9aaaf None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:16.860704 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:16.888924 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d8f46-dbeb-4672-8ff0-d8856fd70f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:16.890793 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d8f46-dbeb-4672-8ff0-d8856fd70f55 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:16.891256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d8f46-dbeb-4672-8ff0-d8856fd70f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:16.891909 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d8f46-dbeb-4672-8ff0-d8856fd70f55 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:16.893004 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2d8f46-dbeb-4672-8ff0-d8856fd70f55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:16.893467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2d8f46-dbeb-4672-8ff0-d8856fd70f55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:16.895121 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2d8f46-dbeb-4672-8ff0-d8856fd70f55 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:16.911172 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 363/715] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 21:18:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:16.912587 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d8f46-dbeb-4672-8ff0-d8856fd70f55 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:16.944520 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9803e87f-958a-4025-ac64-ea4da56d33d8 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:17.011689 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c0235-7c0d-4324-a276-f18a41bc22f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:17.013552 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c0235-7c0d-4324-a276-f18a41bc22f5 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:17.013689 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c0235-7c0d-4324-a276-f18a41bc22f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:17.014350 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c0235-7c0d-4324-a276-f18a41bc22f5 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:17.015871 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66c0235-7c0d-4324-a276-f18a41bc22f5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:17.016620 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66c0235-7c0d-4324-a276-f18a41bc22f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:17.019501 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66c0235-7c0d-4324-a276-f18a41bc22f5 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:17.038279 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 353/716] 10.222.0.65 () {56 vars in 1124 bytes} [Thu May 19 21:18:16 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:17.038279 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c0235-7c0d-4324-a276-f18a41bc22f5 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:17.084657 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d8f46-dbeb-4672-8ff0-d8856fd70f55 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:17.111969 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:17.142107 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ba86c-2598-41bc-9a41-f0cc8dfb40eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:17.143284 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ba86c-2598-41bc-9a41-f0cc8dfb40eb None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:17.143799 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ba86c-2598-41bc-9a41-f0cc8dfb40eb None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:17.144322 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ba86c-2598-41bc-9a41-f0cc8dfb40eb None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:17.145290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9ba86c-2598-41bc-9a41-f0cc8dfb40eb None admin] RBAC: Adding query filter params (name=network) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:17.145824 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9ba86c-2598-41bc-9a41-f0cc8dfb40eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:17.147838 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9ba86c-2598-41bc-9a41-f0cc8dfb40eb None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:17.154790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 364/717] 10.222.0.65 () {56 vars in 1112 bytes} [Thu May 19 21:18:16 2022] GET /identity/v3/services?name=network => generated 124 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:17.174574 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ba86c-2598-41bc-9a41-f0cc8dfb40eb None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:17.307964 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ba86c-2598-41bc-9a41-f0cc8dfb40eb None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:17.354145 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c540480b-60c8-4576-aed9-8ae713aa71db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:17.355314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c540480b-60c8-4576-aed9-8ae713aa71db None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:17.356351 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c540480b-60c8-4576-aed9-8ae713aa71db None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:17.356877 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c540480b-60c8-4576-aed9-8ae713aa71db None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:17.357872 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c540480b-60c8-4576-aed9-8ae713aa71db None admin] RBAC: Adding query filter params (type=network) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:17.358436 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c540480b-60c8-4576-aed9-8ae713aa71db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:17.360271 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c540480b-60c8-4576-aed9-8ae713aa71db None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:17.369525 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 365/718] 10.222.0.65 () {56 vars in 1112 bytes} [Thu May 19 21:18:17 2022] GET /identity/v3/services?type=network => generated 124 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:17.394624 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c0235-7c0d-4324-a276-f18a41bc22f5 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:17.435993 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:17.448655 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c30f3ba-14bb-4a92-b0c9-42a2b80be6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:17.449668 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30f3ba-14bb-4a92-b0c9-42a2b80be6d4 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:17.450132 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30f3ba-14bb-4a92-b0c9-42a2b80be6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:17.450623 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30f3ba-14bb-4a92-b0c9-42a2b80be6d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:17.451704 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c30f3ba-14bb-4a92-b0c9-42a2b80be6d4 None admin] RBAC: Adding query filter params (role.id=2b714f2e5c6c469f8fa037bda1ed16f6, scope.project.id=c4940befa9c4405699c72deedc8a2127, user.id=e254671b21fe4661a381b9521810cf80) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:17.452174 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c30f3ba-14bb-4a92-b0c9-42a2b80be6d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:17.453819 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c30f3ba-14bb-4a92-b0c9-42a2b80be6d4 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:17.461443 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 354/719] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:18:17 2022] GET /identity/v3/role_assignments?user.id=e254671b21fe4661a381b9521810cf80&scope.project.id=c4940befa9c4405699c72deedc8a2127&role.id=2b714f2e5c6c469f8fa037bda1ed16f6 => generated 259 bytes in 429 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:17.816582 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e086b4-69c9-42c6-8f5d-c1fb327afd2e None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:17.817018 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e086b4-69c9-42c6-8f5d-c1fb327afd2e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:17.817454 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e086b4-69c9-42c6-8f5d-c1fb327afd2e None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:17.820470 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 366/720] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:18:17.827856 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd1ea6-0d1a-4d06-bd16-89262054e0c7 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:17.828356 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd1ea6-0d1a-4d06-bd16-89262054e0c7 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:17.828855 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd1ea6-0d1a-4d06-bd16-89262054e0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:17.853274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-fdcd1ea6-0d1a-4d06-bd16-89262054e0c7 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:17.854696 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-fdcd1ea6-0d1a-4d06-bd16-89262054e0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:17.956761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 355/721] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:18:17.980551 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e086b4-69c9-42c6-8f5d-c1fb327afd2e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:18.094231 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:18.176192 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e086b4-69c9-42c6-8f5d-c1fb327afd2e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:18.225765 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2bec8-597c-4715-bce6-d2351b85d436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:18.226883 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2bec8-597c-4715-bce6-d2351b85d436 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:18.227182 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2bec8-597c-4715-bce6-d2351b85d436 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:18.227519 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2bec8-597c-4715-bce6-d2351b85d436 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:18.251210 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe2bec8-597c-4715-bce6-d2351b85d436 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:18:18.259322 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-cbe2bec8-597c-4715-bce6-d2351b85d436 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:18:18.259322 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:18.259322 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:18.259322 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:18.259322 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:18.259322 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:18.259322 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:18.259322 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:18.259322 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:18.259322 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:18.259322 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:18.259322 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:18.259322 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:18:18.259322 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:18:18.259322 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:18:18.259322 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:18:18.261093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:18:18.261093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:18:18.261093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:18:18.261093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:18:18.261093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:18.261093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:18.261093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:18.261093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:18.261093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:18.261093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:18.261093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:18.261093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:18.261093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:18.261093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:18.261093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:18.261093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:18.262673 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:18.262673 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:18.262673 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:18.262673 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:18.262673 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:18:18.262673 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:18:18.262673 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:18:18.262673 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:18:18.262673 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:18:18.262673 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:18:18.262673 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:18:18.262673 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:18.262673 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 367/722] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 21:18:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:18.268848 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcd1ea6-0d1a-4d06-bd16-89262054e0c7 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:18.497703 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcd1ea6-0d1a-4d06-bd16-89262054e0c7 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:18.541753 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:18.551947 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c997e3-f998-49c9-bd9f-f34ca2625442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:18.555214 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c997e3-f998-49c9-bd9f-f34ca2625442 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:18.555497 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c997e3-f998-49c9-bd9f-f34ca2625442 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:18.555777 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c997e3-f998-49c9-bd9f-f34ca2625442 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:18.556572 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c997e3-f998-49c9-bd9f-f34ca2625442 None admin] RBAC: Adding query filter params (name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:18.556879 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c997e3-f998-49c9-bd9f-f34ca2625442 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:18.558367 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c997e3-f998-49c9-bd9f-f34ca2625442 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:18.570658 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 356/723] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 21:18:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:18.577473 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2bec8-597c-4715-bce6-d2351b85d436 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:18.834543 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:18.847102 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2bec8-597c-4715-bce6-d2351b85d436 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:18.927883 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550d3ae1-f3cf-49bc-b64a-0909ad457ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:18.929277 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d3ae1-f3cf-49bc-b64a-0909ad457ebd None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:18.929277 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d3ae1-f3cf-49bc-b64a-0909ad457ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:18.929277 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d3ae1-f3cf-49bc-b64a-0909ad457ebd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:18.948130 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550d3ae1-f3cf-49bc-b64a-0909ad457ebd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:18.949658 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550d3ae1-f3cf-49bc-b64a-0909ad457ebd None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:18.978564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-550d3ae1-f3cf-49bc-b64a-0909ad457ebd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:18.978564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:18.978564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:18.978564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:18.978564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:18.978564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:18.978564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:18.978564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:18.978564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:18.978564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:18.978564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:18.978564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:18.978564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:18.978564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:18.978564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:18.982320 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:18.982320 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:18.982320 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:18.982320 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:18.982320 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:18.982320 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:18.982320 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:18.982320 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:18.982320 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:18.982320 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:18.982320 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:18.982320 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:18.982320 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:18.982320 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:18.982320 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:18.982320 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:18.982320 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:18.984137 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:18.984137 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:18.984137 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:18.984137 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:18.984137 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 368/724] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 404 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:18:18.989085 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c997e3-f998-49c9-bd9f-f34ca2625442 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:19.142278 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c997e3-f998-49c9-bd9f-f34ca2625442 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:19.194892 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d625a-0771-4263-8ec8-b1dde605ff8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:19.195491 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d625a-0771-4263-8ec8-b1dde605ff8f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:19.195491 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d625a-0771-4263-8ec8-b1dde605ff8f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:19.195997 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d625a-0771-4263-8ec8-b1dde605ff8f None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:19.196462 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595d625a-0771-4263-8ec8-b1dde605ff8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:19.196768 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595d625a-0771-4263-8ec8-b1dde605ff8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:19.198169 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595d625a-0771-4263-8ec8-b1dde605ff8f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:19.219962 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 357/725] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:18:19.228447 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550d3ae1-f3cf-49bc-b64a-0909ad457ebd None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:19.451426 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550d3ae1-f3cf-49bc-b64a-0909ad457ebd None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:19.545171 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c2cc6-adce-40e8-80df-42e419f34900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:19.546568 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c2cc6-adce-40e8-80df-42e419f34900 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:19.547057 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c2cc6-adce-40e8-80df-42e419f34900 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:19.547532 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c2cc6-adce-40e8-80df-42e419f34900 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:19.566223 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c2cc6-adce-40e8-80df-42e419f34900 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:19.569398 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c2cc6-adce-40e8-80df-42e419f34900 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:19.606266 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-605c2cc6-adce-40e8-80df-42e419f34900 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 21:18:19.606266 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:19.606266 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:19.606266 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:19.606266 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:19.606266 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:19.606266 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:19.606266 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:19.606266 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:19.606266 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:19.606266 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:19.606266 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:19.606266 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:18:19.606266 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:18:19.606266 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:18:19.606266 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:18:19.609762 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:19.609762 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:19.609762 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:18:19.609762 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:19.609762 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:18:19.609762 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:19.609762 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:19.609762 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:19.609762 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:19.609762 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:19.609762 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:19.609762 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:19.609762 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:19.609762 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:19.609762 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:19.609762 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:19.609762 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:19.609762 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:19.613471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:19.613471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:19.613471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:18:19.613471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:18:19.613471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:18:19.613471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:18:19.613471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:18:19.613471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:18:19.613471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 21:18:19.613471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:19.618275 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 369/726] 10.222.0.65 () {56 vars in 1117 bytes} [Thu May 19 21:18:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:19.626126 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d625a-0771-4263-8ec8-b1dde605ff8f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:19.672370 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d41dd-58fd-407a-9a2f-0e7335fb2a5b None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:19.672817 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d41dd-58fd-407a-9a2f-0e7335fb2a5b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:19.673281 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d41dd-58fd-407a-9a2f-0e7335fb2a5b None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:19.676178 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 370/727] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:18:19.683241 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def45dc6-a4aa-48f8-84c2-0edd15655c24 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:19.683700 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def45dc6-a4aa-48f8-84c2-0edd15655c24 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:19.684184 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def45dc6-a4aa-48f8-84c2-0edd15655c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:19.726368 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-def45dc6-a4aa-48f8-84c2-0edd15655c24 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:19.728275 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-def45dc6-a4aa-48f8-84c2-0edd15655c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:19.747266 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:19.788722 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:19.887163 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d625a-0771-4263-8ec8-b1dde605ff8f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:19.963624 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5bfbd3-b5a9-41bb-815b-d32005282555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:19.964706 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5bfbd3-b5a9-41bb-815b-d32005282555 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:19.965248 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5bfbd3-b5a9-41bb-815b-d32005282555 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:19.965708 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5bfbd3-b5a9-41bb-815b-d32005282555 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:19.967310 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5bfbd3-b5a9-41bb-815b-d32005282555 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:19.967814 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5bfbd3-b5a9-41bb-815b-d32005282555 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:19.970449 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5bfbd3-b5a9-41bb-815b-d32005282555 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:20.016860 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 371/728] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:18:20.033661 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 358/729] 10.222.0.65 () {56 vars in 1149 bytes} [Thu May 19 21:18:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:20.042601 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def45dc6-a4aa-48f8-84c2-0edd15655c24 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:20.087313 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5bfbd3-b5a9-41bb-815b-d32005282555 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:20.256985 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def45dc6-a4aa-48f8-84c2-0edd15655c24 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:20.301064 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4634f22a-b6ee-489d-956d-c372ff762713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:20.301633 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634f22a-b6ee-489d-956d-c372ff762713 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:20.301907 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634f22a-b6ee-489d-956d-c372ff762713 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:20.302181 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634f22a-b6ee-489d-956d-c372ff762713 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:20.341933 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:20.482943 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5bfbd3-b5a9-41bb-815b-d32005282555 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:20.517375 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:20.525992 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4634f22a-b6ee-489d-956d-c372ff762713 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:20.525992 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4634f22a-b6ee-489d-956d-c372ff762713 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:20.579374 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d427581d-c90f-452b-8faa-a6648d222be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:20.580105 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427581d-c90f-452b-8faa-a6648d222be2 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:20.580379 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427581d-c90f-452b-8faa-a6648d222be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:20.580664 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427581d-c90f-452b-8faa-a6648d222be2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:20.608324 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d427581d-c90f-452b-8faa-a6648d222be2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:18:20.623309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-d427581d-c90f-452b-8faa-a6648d222be2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:18:20.623309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:20.623309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:20.623309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:20.623309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:20.623309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:20.623309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:20.623309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:20.623309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:20.623309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:20.623309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:20.623309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:20.623309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:18:20.623309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:18:20.623309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:18:20.623309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:18:20.625030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:18:20.625030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:18:20.625030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:18:20.625030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:18:20.625030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:20.625030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:20.625030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:20.625030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:20.625030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:20.625030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:20.625030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:20.625030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:20.625030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:20.625030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:20.625030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:20.625030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:20.627925 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:20.627925 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:20.627925 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:20.627925 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:20.627925 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:18:20.627925 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:18:20.627925 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:18:20.627925 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:18:20.627925 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:18:20.627925 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:18:20.627925 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:18:20.627925 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:20.627925 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 359/730] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 21:18:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 539 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:18:20.635698 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d427581d-c90f-452b-8faa-a6648d222be2 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:20.735417 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 372/731] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:18:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 694 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 21:18:20.744595 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4634f22a-b6ee-489d-956d-c372ff762713 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:20.824926 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:20.830704 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d427581d-c90f-452b-8faa-a6648d222be2 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:20.871233 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a4e1a-c5f8-4596-8455-b1c61c341703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:20.871233 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a4e1a-c5f8-4596-8455-b1c61c341703 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:20.871233 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a4e1a-c5f8-4596-8455-b1c61c341703 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:20.871778 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a4e1a-c5f8-4596-8455-b1c61c341703 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:20.872251 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a4e1a-c5f8-4596-8455-b1c61c341703 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:20.872725 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a4e1a-c5f8-4596-8455-b1c61c341703 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:20.874427 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a4e1a-c5f8-4596-8455-b1c61c341703 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:20.884660 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 360/732] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:18:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:18:20.891773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff7f2f1-354c-4170-b9e3-d62e21153890 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:20.892589 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff7f2f1-354c-4170-b9e3-d62e21153890 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:20.893252 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff7f2f1-354c-4170-b9e3-d62e21153890 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:20.896526 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 361/733] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:20 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:18:20.898734 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff7f2f1-354c-4170-b9e3-d62e21153890 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:20.932699 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:20.938493 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4634f22a-b6ee-489d-956d-c372ff762713 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:21.002786 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc540e-0e1a-4421-b05a-f8e9cc0e3f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:21.004568 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc540e-0e1a-4421-b05a-f8e9cc0e3f40 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:21.005314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc540e-0e1a-4421-b05a-f8e9cc0e3f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:21.006050 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc540e-0e1a-4421-b05a-f8e9cc0e3f40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:21.026699 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dc540e-0e1a-4421-b05a-f8e9cc0e3f40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:21.030355 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dc540e-0e1a-4421-b05a-f8e9cc0e3f40 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:21.030602 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff7f2f1-354c-4170-b9e3-d62e21153890 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:21.071932 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-e2dc540e-0e1a-4421-b05a-f8e9cc0e3f40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:21.071932 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:21.071932 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:21.071932 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:21.071932 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:21.071932 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:21.071932 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:21.071932 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:21.071932 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:21.071932 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:21.071932 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:21.071932 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:21.071932 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:21.071932 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:21.071932 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:21.073989 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:21.073989 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:21.073989 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:21.073989 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:21.073989 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:21.073989 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:21.073989 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:21.073989 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:21.073989 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:21.073989 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:21.073989 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:21.073989 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:21.073989 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:21.073989 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:21.073989 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:21.073989 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:21.073989 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:21.083967 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:21.083967 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:21.083967 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:21.083967 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:21.084506 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d56e98-c881-4526-b3d8-462dd36ad96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:21.085162 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 373/734] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:21.085295 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d56e98-c881-4526-b3d8-462dd36ad96e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:21.085424 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f215813-4c69-489c-89ca-4b6712a6d657 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:21.085538 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d56e98-c881-4526-b3d8-462dd36ad96e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:21.085648 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f215813-4c69-489c-89ca-4b6712a6d657 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:21.085783 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d56e98-c881-4526-b3d8-462dd36ad96e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:21.085911 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f215813-4c69-489c-89ca-4b6712a6d657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:21.096567 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-4f215813-4c69-489c-89ca-4b6712a6d657 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:21.097760 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4f215813-4c69-489c-89ca-4b6712a6d657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:21.103645 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d56e98-c881-4526-b3d8-462dd36ad96e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:21.105543 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d56e98-c881-4526-b3d8-462dd36ad96e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:21.127168 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-f6d56e98-c881-4526-b3d8-462dd36ad96e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:21.127168 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:21.127168 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:21.127168 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:21.127168 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:21.127168 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:21.127168 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:21.127168 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:21.127168 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:21.127168 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:21.127168 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:21.127168 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:21.127168 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:21.127168 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:21.127168 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:21.128873 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:21.128873 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:21.128873 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:21.128873 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:21.128873 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:21.128873 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:21.128873 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:21.128873 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:21.128873 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:21.128873 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:21.128873 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:21.128873 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:21.128873 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:21.128873 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:21.128873 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:21.128873 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:21.128873 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:21.130599 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:21.130599 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:21.130599 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:21.130599 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:21.130599 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 362/735] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:21.133944 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8355c-bdb6-4506-ad15-2bc950c43edc None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:21.133944 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8355c-bdb6-4506-ad15-2bc950c43edc None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:21.133944 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8355c-bdb6-4506-ad15-2bc950c43edc None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:21.138482 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 363/736] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:18:21.138482 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8355c-bdb6-4506-ad15-2bc950c43edc None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:21.238329 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 374/737] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:18:21.238976 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f215813-4c69-489c-89ca-4b6712a6d657 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:21.254094 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:21.287772 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:21.294408 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8355c-bdb6-4506-ad15-2bc950c43edc None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:21.355940 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22359bef-b8b4-4efa-bf52-02b9e345bf69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:21.356628 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22359bef-b8b4-4efa-bf52-02b9e345bf69 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:21.356907 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22359bef-b8b4-4efa-bf52-02b9e345bf69 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:21.357180 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22359bef-b8b4-4efa-bf52-02b9e345bf69 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:21.358179 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22359bef-b8b4-4efa-bf52-02b9e345bf69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:21.358527 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22359bef-b8b4-4efa-bf52-02b9e345bf69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:21.361044 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22359bef-b8b4-4efa-bf52-02b9e345bf69 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:21.382296 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 364/738] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:21.382296 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8078efe-6caf-4f8e-9e85-a6b5e81cdfd9 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:21.382296 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8078efe-6caf-4f8e-9e85-a6b5e81cdfd9 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:21.382296 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8078efe-6caf-4f8e-9e85-a6b5e81cdfd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:21.404434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-a8078efe-6caf-4f8e-9e85-a6b5e81cdfd9 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:21.405926 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a8078efe-6caf-4f8e-9e85-a6b5e81cdfd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:21.413199 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f215813-4c69-489c-89ca-4b6712a6d657 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:21.479915 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58396ef8-018e-4674-a348-bf4f2cad8472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:21.481066 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58396ef8-018e-4674-a348-bf4f2cad8472 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:21.481552 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58396ef8-018e-4674-a348-bf4f2cad8472 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:21.482041 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58396ef8-018e-4674-a348-bf4f2cad8472 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:21.483582 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58396ef8-018e-4674-a348-bf4f2cad8472 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:21.484132 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58396ef8-018e-4674-a348-bf4f2cad8472 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:21.486935 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58396ef8-018e-4674-a348-bf4f2cad8472 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:21.505135 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 365/739] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 21:18:21.506468 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8078efe-6caf-4f8e-9e85-a6b5e81cdfd9 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:21.517765 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 375/740] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:21.519770 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58396ef8-018e-4674-a348-bf4f2cad8472 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:21.621375 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:21.677404 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8078efe-6caf-4f8e-9e85-a6b5e81cdfd9 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:21.738964 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e7b572-96a8-4429-afc0-3895dfc50dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:21.740006 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e7b572-96a8-4429-afc0-3895dfc50dd5 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:21.740567 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e7b572-96a8-4429-afc0-3895dfc50dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:21.742182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e7b572-96a8-4429-afc0-3895dfc50dd5 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:21.743390 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e7b572-96a8-4429-afc0-3895dfc50dd5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:21.745622 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e7b572-96a8-4429-afc0-3895dfc50dd5 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:21.778287 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 366/741] 10.222.0.65 () {60 vars in 1141 bytes} [Thu May 19 21:18:21 2022] POST /identity/v3/services => generated 240 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:18:21.778287 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e7b572-96a8-4429-afc0-3895dfc50dd5 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:21.802331 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:21.821761 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58396ef8-018e-4674-a348-bf4f2cad8472 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:21.907705 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3464be-a263-4524-9b01-bf320b730b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:21.908984 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3464be-a263-4524-9b01-bf320b730b75 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:21.909471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3464be-a263-4524-9b01-bf320b730b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:21.909964 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3464be-a263-4524-9b01-bf320b730b75 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:21.924009 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3464be-a263-4524-9b01-bf320b730b75 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:21.926904 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3464be-a263-4524-9b01-bf320b730b75 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:21.935220 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e7b572-96a8-4429-afc0-3895dfc50dd5 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:21.953678 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-4e3464be-a263-4524-9b01-bf320b730b75 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:21.953678 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:21.953678 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:21.953678 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:21.953678 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:21.953678 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:21.953678 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:21.953678 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:21.953678 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:21.953678 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:21.953678 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:21.953678 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:21.953678 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:18:21.953678 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:18:21.953678 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:18:21.955306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:18:21.955306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:21.955306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:21.955306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:21.955306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:21.955306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:21.955306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:21.955306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:21.955306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:21.955306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:21.955306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:21.955306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:21.955306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:21.955306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:21.955306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:21.955306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:21.955306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:21.956637 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:18:21.956637 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:18:21.956637 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:18:21.956637 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:18:21.956637 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:18:21.956637 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:18:21.956637 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:21.956637 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:21.956637 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 376/742] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:18:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 437 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:21.958496 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb33c1a-b8ae-43ee-801f-f08140ebede1 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:21.959012 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb33c1a-b8ae-43ee-801f-f08140ebede1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:21.960393 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb33c1a-b8ae-43ee-801f-f08140ebede1 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:21.963330 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 377/743] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:18:21.964471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb33c1a-b8ae-43ee-801f-f08140ebede1 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:21.985351 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:21.998384 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79bac2f-1988-4e95-ac7f-26d533dc2e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:21.999324 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79bac2f-1988-4e95-ac7f-26d533dc2e38 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:21.999611 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79bac2f-1988-4e95-ac7f-26d533dc2e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:21.999890 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79bac2f-1988-4e95-ac7f-26d533dc2e38 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:22.012341 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79bac2f-1988-4e95-ac7f-26d533dc2e38 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:22.014618 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79bac2f-1988-4e95-ac7f-26d533dc2e38 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:22.034005 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-d79bac2f-1988-4e95-ac7f-26d533dc2e38 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 21:18:22.034005 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:22.034005 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:22.034005 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:22.034005 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:22.034005 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:22.034005 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:22.034005 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:22.034005 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:22.034005 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:22.034005 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:22.034005 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:22.034005 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:18:22.034005 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:18:22.034005 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:18:22.034005 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:18:22.035315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:22.035315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:22.035315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:18:22.035315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:22.035315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:18:22.035315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:22.035315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:22.035315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:22.035315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:22.035315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:22.035315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:22.035315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:22.035315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:22.035315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:22.035315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:22.035315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:22.035315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:22.035315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:22.036865 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:22.036865 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:22.036865 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:18:22.036865 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:18:22.036865 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:18:22.036865 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:18:22.036865 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:18:22.036865 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:18:22.036865 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 21:18:22.036865 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:22.036865 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 367/744] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 21:18:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:22.036865 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79bac2f-1988-4e95-ac7f-26d533dc2e38 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:22.134984 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb33c1a-b8ae-43ee-801f-f08140ebede1 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:22.185208 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79bac2f-1988-4e95-ac7f-26d533dc2e38 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:22.192377 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd2b88-e516-43f7-92a8-83f5e0833d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:22.193438 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd2b88-e516-43f7-92a8-83f5e0833d37 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:22.194394 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd2b88-e516-43f7-92a8-83f5e0833d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:22.194885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd2b88-e516-43f7-92a8-83f5e0833d37 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:22.195862 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dd2b88-e516-43f7-92a8-83f5e0833d37 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:22.197482 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dd2b88-e516-43f7-92a8-83f5e0833d37 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:22.211593 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:22.218862 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 378/745] 10.222.0.65 () {60 vars in 1141 bytes} [Thu May 19 21:18:21 2022] POST /identity/v3/services => generated 251 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:18:22.221588 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7eaba-41ed-4965-b0c1-61c77f7d7fd6 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:22.222626 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7eaba-41ed-4965-b0c1-61c77f7d7fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:22.223684 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7eaba-41ed-4965-b0c1-61c77f7d7fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:22.256631 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a8c7eaba-41ed-4965-b0c1-61c77f7d7fd6 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:22.259215 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a8c7eaba-41ed-4965-b0c1-61c77f7d7fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:22.271318 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af79d75-32ea-4034-8d68-1d40986d6a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:22.272783 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af79d75-32ea-4034-8d68-1d40986d6a92 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:22.273469 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af79d75-32ea-4034-8d68-1d40986d6a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:22.274124 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af79d75-32ea-4034-8d68-1d40986d6a92 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:22.276063 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af79d75-32ea-4034-8d68-1d40986d6a92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:22.277899 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af79d75-32ea-4034-8d68-1d40986d6a92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:22.280499 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af79d75-32ea-4034-8d68-1d40986d6a92 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:22.315222 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 368/746] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 21:18:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:22.317142 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af79d75-32ea-4034-8d68-1d40986d6a92 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:22.406834 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 379/747] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:18:22.408353 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7eaba-41ed-4965-b0c1-61c77f7d7fd6 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:22.420047 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:22.528693 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af79d75-32ea-4034-8d68-1d40986d6a92 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:22.566434 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7eaba-41ed-4965-b0c1-61c77f7d7fd6 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:22.573676 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8c2a9-487b-469a-bf71-52c9c1d9b256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:22.575166 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8c2a9-487b-469a-bf71-52c9c1d9b256 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:22.575690 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8c2a9-487b-469a-bf71-52c9c1d9b256 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:22.576223 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8c2a9-487b-469a-bf71-52c9c1d9b256 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:22.577382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a8c2a9-487b-469a-bf71-52c9c1d9b256 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:22.577906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a8c2a9-487b-469a-bf71-52c9c1d9b256 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:22.579571 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a8c2a9-487b-469a-bf71-52c9c1d9b256 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:22.602978 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 369/748] 10.222.0.65 () {56 vars in 1135 bytes} [Thu May 19 21:18:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:22.608509 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8c2a9-487b-469a-bf71-52c9c1d9b256 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:22.623603 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:22.633694 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0fcbba-52cf-4371-b280-eb8e6496de31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:22.634985 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0fcbba-52cf-4371-b280-eb8e6496de31 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:22.635568 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0fcbba-52cf-4371-b280-eb8e6496de31 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:22.636142 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0fcbba-52cf-4371-b280-eb8e6496de31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:22.637674 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0fcbba-52cf-4371-b280-eb8e6496de31 None admin] RBAC: Adding query filter params (role.id=9dc26e65165b49d6916b54458f847755, scope.project.id=c4940befa9c4405699c72deedc8a2127, user.id=eba91dff7cac401e81f6fb6391874af1) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:22.638376 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0fcbba-52cf-4371-b280-eb8e6496de31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:22.641026 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0fcbba-52cf-4371-b280-eb8e6496de31 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:22.658424 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 380/749] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:18:22 2022] GET /identity/v3/role_assignments?user.id=eba91dff7cac401e81f6fb6391874af1&scope.project.id=c4940befa9c4405699c72deedc8a2127&role.id=9dc26e65165b49d6916b54458f847755 => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:22.660223 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0fcbba-52cf-4371-b280-eb8e6496de31 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:22.739493 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8c2a9-487b-469a-bf71-52c9c1d9b256 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:22.743207 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:22.788277 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab651fb2-c0cc-4bd2-bc4f-458b2d2f90eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:22.789501 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab651fb2-c0cc-4bd2-bc4f-458b2d2f90eb None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:22.789946 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab651fb2-c0cc-4bd2-bc4f-458b2d2f90eb None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:22.790407 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab651fb2-c0cc-4bd2-bc4f-458b2d2f90eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:22.819716 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab651fb2-c0cc-4bd2-bc4f-458b2d2f90eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:18:22.828774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-ab651fb2-c0cc-4bd2-bc4f-458b2d2f90eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:18:22.828774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:22.828774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:22.828774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:22.828774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:22.828774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:22.828774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:22.828774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:22.828774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:22.828774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:22.828774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:22.828774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:22.828774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:18:22.828774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:18:22.828774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:18:22.828774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:18:22.829897 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:18:22.829897 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:18:22.829897 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:18:22.829897 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:18:22.829897 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:22.829897 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:22.829897 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:22.829897 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:22.829897 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:22.829897 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:22.829897 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:22.829897 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:22.829897 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:22.829897 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:22.829897 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:22.829897 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:22.830835 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:22.830835 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:22.830835 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:22.830835 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:22.830835 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:18:22.830835 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:18:22.830835 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:18:22.830835 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:18:22.830835 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:18:22.830835 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:18:22.830835 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:18:22.830835 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:22.840427 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 370/750] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 21:18:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:22.847389 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab651fb2-c0cc-4bd2-bc4f-458b2d2f90eb None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:22.935521 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0fcbba-52cf-4371-b280-eb8e6496de31 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:22.983453 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b53a60-2477-40b6-9c8d-f121be0d5cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:22.984160 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b53a60-2477-40b6-9c8d-f121be0d5cb8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:22.984454 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b53a60-2477-40b6-9c8d-f121be0d5cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:22.984735 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b53a60-2477-40b6-9c8d-f121be0d5cb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:23.045511 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab651fb2-c0cc-4bd2-bc4f-458b2d2f90eb None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:23.066852 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:23.078607 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b53a60-2477-40b6-9c8d-f121be0d5cb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:23.080556 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b53a60-2477-40b6-9c8d-f121be0d5cb8 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:23.087007 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59cf5a-0d23-4bb8-9fe3-a2780406fc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:23.087996 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59cf5a-0d23-4bb8-9fe3-a2780406fc66 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:23.088421 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59cf5a-0d23-4bb8-9fe3-a2780406fc66 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:23.088858 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59cf5a-0d23-4bb8-9fe3-a2780406fc66 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:23.089758 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f59cf5a-0d23-4bb8-9fe3-a2780406fc66 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:23.090207 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f59cf5a-0d23-4bb8-9fe3-a2780406fc66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:23.091837 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f59cf5a-0d23-4bb8-9fe3-a2780406fc66 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:23.102310 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 371/751] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:18:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:23.108993 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59cf5a-0d23-4bb8-9fe3-a2780406fc66 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:23.175101 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:23.336448 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 381/752] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:18:22 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 677 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 21:18:23.339798 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59cf5a-0d23-4bb8-9fe3-a2780406fc66 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:23.341791 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b53a60-2477-40b6-9c8d-f121be0d5cb8 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:23.431779 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d6583-a317-4f55-9501-d7e86c74b899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:23.432637 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d6583-a317-4f55-9501-d7e86c74b899 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:23.432919 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d6583-a317-4f55-9501-d7e86c74b899 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:23.433200 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d6583-a317-4f55-9501-d7e86c74b899 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:23.453775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21d6583-a317-4f55-9501-d7e86c74b899 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:23.455292 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21d6583-a317-4f55-9501-d7e86c74b899 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:23.477813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-f21d6583-a317-4f55-9501-d7e86c74b899 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:23.477813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:23.477813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:23.477813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:23.477813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:23.477813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:23.477813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:23.477813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:23.477813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:23.477813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:23.477813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:23.477813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:23.477813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:23.477813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:23.477813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:23.478853 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:23.478853 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:23.478853 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:23.478853 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:23.478853 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:23.478853 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:23.478853 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:23.478853 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:23.478853 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:23.478853 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:23.478853 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:23.478853 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:23.478853 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:23.478853 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:23.478853 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:23.478853 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:23.478853 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:23.479734 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:23.479734 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:23.479734 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:23.479734 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:23.479734 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 372/753] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:23.488463 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d6583-a317-4f55-9501-d7e86c74b899 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:23.513078 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:23.521982 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b53a60-2477-40b6-9c8d-f121be0d5cb8 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:23.574400 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c05c44-25d1-44db-a8da-b1d51e530035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:23.575747 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c05c44-25d1-44db-a8da-b1d51e530035 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:23.576320 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c05c44-25d1-44db-a8da-b1d51e530035 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:23.576805 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c05c44-25d1-44db-a8da-b1d51e530035 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:23.592338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c05c44-25d1-44db-a8da-b1d51e530035 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:23.594191 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c05c44-25d1-44db-a8da-b1d51e530035 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:23.615009 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-57c05c44-25d1-44db-a8da-b1d51e530035 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:23.615009 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:23.615009 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:23.615009 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:23.615009 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:23.615009 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:23.615009 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:23.615009 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:23.615009 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:23.615009 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:23.615009 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:23.615009 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:23.615009 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:23.615009 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:23.615009 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:23.616367 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:23.616367 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:23.616367 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:23.616367 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:23.616367 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:23.616367 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:23.616367 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:23.616367 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:23.616367 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:23.616367 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:23.616367 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:23.616367 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:23.616367 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:23.616367 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:23.616367 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:23.616367 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:23.616367 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:23.617258 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:23.617258 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:23.617258 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:23.617258 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:23.619986 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 382/754] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:18:23.625820 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c05c44-25d1-44db-a8da-b1d51e530035 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:23.722167 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:23.766747 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d6583-a317-4f55-9501-d7e86c74b899 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:23.798760 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c05c44-25d1-44db-a8da-b1d51e530035 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:23.837094 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d5fab-114a-4315-876c-946db0140ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:23.838437 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d5fab-114a-4315-876c-946db0140ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:23.838967 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d5fab-114a-4315-876c-946db0140ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:23.839466 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d5fab-114a-4315-876c-946db0140ae6 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:23.840411 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d5fab-114a-4315-876c-946db0140ae6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:23.840920 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d5fab-114a-4315-876c-946db0140ae6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:23.843126 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d5fab-114a-4315-876c-946db0140ae6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:23.862779 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 383/755] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:23.871998 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d5fab-114a-4315-876c-946db0140ae6 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:23.889211 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:23.895138 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a3c02-bf9b-474d-8777-08efe4a3f91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:23.895837 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a3c02-bf9b-474d-8777-08efe4a3f91a None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:23.896117 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a3c02-bf9b-474d-8777-08efe4a3f91a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:23.896390 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a3c02-bf9b-474d-8777-08efe4a3f91a None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:23.897194 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036a3c02-bf9b-474d-8777-08efe4a3f91a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:23.897912 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036a3c02-bf9b-474d-8777-08efe4a3f91a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:23.898924 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036a3c02-bf9b-474d-8777-08efe4a3f91a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:23.930636 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 373/756] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 442 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:23.940266 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a3c02-bf9b-474d-8777-08efe4a3f91a None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:24.151883 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d5fab-114a-4315-876c-946db0140ae6 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:24.247778 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b3c3a7-d2ab-4200-a604-d525cca80268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:24.249152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b3c3a7-d2ab-4200-a604-d525cca80268 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:24.249632 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b3c3a7-d2ab-4200-a604-d525cca80268 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:24.250110 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b3c3a7-d2ab-4200-a604-d525cca80268 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:24.264388 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a3c02-bf9b-474d-8777-08efe4a3f91a None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:24.267158 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b3c3a7-d2ab-4200-a604-d525cca80268 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:24.269835 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b3c3a7-d2ab-4200-a604-d525cca80268 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:24.306358 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-52b3c3a7-d2ab-4200-a604-d525cca80268 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:24.306358 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:24.306358 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:24.306358 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:24.306358 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:24.306358 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:24.306358 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:24.306358 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:24.306358 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:24.306358 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:24.306358 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:24.306358 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:24.306358 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:18:24.306358 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:18:24.306358 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:18:24.308493 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:18:24.308493 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:24.308493 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:24.308493 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:24.308493 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:24.308493 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:24.308493 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:24.308493 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:24.308493 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:24.308493 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:24.308493 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:24.308493 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:24.308493 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:24.308493 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:24.308493 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:24.308493 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:24.308493 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:24.310561 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:18:24.310561 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:18:24.310561 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:18:24.310561 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:18:24.310561 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:18:24.310561 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:18:24.310561 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:24.310561 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:24.310561 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 384/757] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:18:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 439 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:24.315801 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d7f62-e345-4dc9-85b4-099ee30818ab None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:24.316163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d7f62-e345-4dc9-85b4-099ee30818ab None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:24.316513 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d7f62-e345-4dc9-85b4-099ee30818ab None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:24.322486 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 385/758] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:18:24.323533 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4d7f62-e345-4dc9-85b4-099ee30818ab None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:24.326142 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772eb074-d500-4b27-80ee-bd4149d3271e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:24.328053 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772eb074-d500-4b27-80ee-bd4149d3271e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:24.329954 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772eb074-d500-4b27-80ee-bd4149d3271e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:24.330511 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772eb074-d500-4b27-80ee-bd4149d3271e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:24.350455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772eb074-d500-4b27-80ee-bd4149d3271e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:24.354757 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772eb074-d500-4b27-80ee-bd4149d3271e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:24.360553 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:24.398211 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-772eb074-d500-4b27-80ee-bd4149d3271e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 21:18:24.398211 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:24.398211 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:24.398211 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:24.398211 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:24.398211 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:24.398211 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:24.398211 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:24.398211 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:24.398211 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:24.398211 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:24.398211 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:24.398211 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:18:24.398211 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:18:24.398211 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:18:24.398211 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:18:24.400574 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:24.400574 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:24.400574 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:18:24.400574 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:24.400574 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:18:24.400574 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:24.400574 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:24.400574 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:24.400574 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:24.400574 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:24.400574 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:24.400574 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:24.400574 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:24.400574 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:24.400574 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:24.400574 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:24.400574 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:24.400574 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:24.402761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:24.402761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:24.402761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:18:24.402761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:18:24.402761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:18:24.402761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:18:24.402761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:18:24.402761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:18:24.402761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 21:18:24.402761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:24.402761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 374/759] 10.222.0.65 () {56 vars in 1093 bytes} [Thu May 19 21:18:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 461 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:24.410276 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad73543d-b728-4ed0-87d0-60884c8a17fa None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:24.410276 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad73543d-b728-4ed0-87d0-60884c8a17fa None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:24.410276 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad73543d-b728-4ed0-87d0-60884c8a17fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:24.422578 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:24.441841 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-ad73543d-b728-4ed0-87d0-60884c8a17fa None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:24.444846 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ad73543d-b728-4ed0-87d0-60884c8a17fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:24.511062 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4d7f62-e345-4dc9-85b4-099ee30818ab None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:24.563776 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d099951-8c3d-40bf-9123-2c335d4427ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:24.565345 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d099951-8c3d-40bf-9123-2c335d4427ec None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:24.567111 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d099951-8c3d-40bf-9123-2c335d4427ec None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:24.567926 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d099951-8c3d-40bf-9123-2c335d4427ec None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:24.570845 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d099951-8c3d-40bf-9123-2c335d4427ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:24.572337 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d099951-8c3d-40bf-9123-2c335d4427ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:24.576610 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d099951-8c3d-40bf-9123-2c335d4427ec None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:24.594429 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 375/760] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:18:24.595588 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad73543d-b728-4ed0-87d0-60884c8a17fa None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:24.607710 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 386/761] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 21:18:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:24.619884 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d099951-8c3d-40bf-9123-2c335d4427ec None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:24.770594 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d099951-8c3d-40bf-9123-2c335d4427ec None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:24.803902 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:24.812598 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad73543d-b728-4ed0-87d0-60884c8a17fa None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:24.819051 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9154967e-7786-47bf-a95e-5b1796dc0da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:24.820601 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9154967e-7786-47bf-a95e-5b1796dc0da8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:24.822181 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9154967e-7786-47bf-a95e-5b1796dc0da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:24.822984 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9154967e-7786-47bf-a95e-5b1796dc0da8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:24.825996 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9154967e-7786-47bf-a95e-5b1796dc0da8 None admin] RBAC: Adding query filter params (role.id=2b714f2e5c6c469f8fa037bda1ed16f6, scope.project.id=c4940befa9c4405699c72deedc8a2127, user.id=7e07984b36934aff9052541ef2a56f36) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:24.827244 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9154967e-7786-47bf-a95e-5b1796dc0da8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:24.831439 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9154967e-7786-47bf-a95e-5b1796dc0da8 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:24.844545 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:24.856119 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 387/762] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:18:24 2022] GET /identity/v3/role_assignments?user.id=7e07984b36934aff9052541ef2a56f36&scope.project.id=c4940befa9c4405699c72deedc8a2127&role.id=2b714f2e5c6c469f8fa037bda1ed16f6 => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:18:24.860128 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9154967e-7786-47bf-a95e-5b1796dc0da8 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:24.865052 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885ca0c4-9290-4c12-afa6-ba6c25f7b8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:24.866762 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ca0c4-9290-4c12-afa6-ba6c25f7b8e4 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:24.867120 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ca0c4-9290-4c12-afa6-ba6c25f7b8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:24.867495 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ca0c4-9290-4c12-afa6-ba6c25f7b8e4 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:24.868520 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885ca0c4-9290-4c12-afa6-ba6c25f7b8e4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:24.868987 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885ca0c4-9290-4c12-afa6-ba6c25f7b8e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:24.870712 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885ca0c4-9290-4c12-afa6-ba6c25f7b8e4 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:24.914493 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 376/763] 10.222.0.65 () {56 vars in 1133 bytes} [Thu May 19 21:18:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:24.922996 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885ca0c4-9290-4c12-afa6-ba6c25f7b8e4 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:25.122974 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9154967e-7786-47bf-a95e-5b1796dc0da8 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:25.136211 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885ca0c4-9290-4c12-afa6-ba6c25f7b8e4 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:25.194230 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88e7cc7-7eb9-4b37-9e3d-463d9ae6b1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:25.195043 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e7cc7-7eb9-4b37-9e3d-463d9ae6b1a2 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:25.195325 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e7cc7-7eb9-4b37-9e3d-463d9ae6b1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:25.195599 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e7cc7-7eb9-4b37-9e3d-463d9ae6b1a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:25.199587 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8659b1-9f1a-483c-aae6-f3457b946cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:25.201315 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8659b1-9f1a-483c-aae6-f3457b946cad None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:25.202036 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8659b1-9f1a-483c-aae6-f3457b946cad None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:25.202722 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8659b1-9f1a-483c-aae6-f3457b946cad None admin] PATH_INFO: `/v3/services/network` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:25.204358 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8659b1-9f1a-483c-aae6-f3457b946cad None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:25.207413 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8659b1-9f1a-483c-aae6-f3457b946cad None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:25.207676 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e7cc7-7eb9-4b37-9e3d-463d9ae6b1a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:25.209074 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e7cc7-7eb9-4b37-9e3d-463d9ae6b1a2 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:25.229781 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-f88e7cc7-7eb9-4b37-9e3d-463d9ae6b1a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:25.229781 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:25.229781 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:25.229781 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:25.229781 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:25.229781 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:25.229781 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:25.229781 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:25.229781 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:25.229781 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:25.229781 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:25.229781 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:25.229781 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:25.229781 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:25.229781 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:25.231871 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:25.231871 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:25.231871 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:25.231871 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:25.231871 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:25.231871 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:25.231871 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:25.231871 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:25.231871 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:25.231871 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:25.231871 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:25.231871 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:25.231871 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:25.231871 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:25.231871 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:25.231871 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:25.231871 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:25.233957 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:25.233957 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:25.233957 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:25.233957 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:25.233957 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 377/764] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:18:25.233957 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68e9e5-9dab-456c-a8cd-ab53f6242b24 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:25.233957 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68e9e5-9dab-456c-a8cd-ab53f6242b24 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:25.233957 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68e9e5-9dab-456c-a8cd-ab53f6242b24 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:25.239293 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 378/765] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:18:25.240307 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e68e9e5-9dab-456c-a8cd-ab53f6242b24 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:25.241324 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-ca8659b1-9f1a-483c-aae6-f3457b946cad None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 21:18:25.241324 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:25.241324 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:18:25.241324 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:18:25.241324 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:18:25.241324 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:18:25.241324 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:18:25.241324 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:18:25.241324 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 21:18:25.241324 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:25.241324 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:18:25.241324 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:25.241324 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:25.241324 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:25.241324 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:25.241324 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:25.241324 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:25.243605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:25.243605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:25.243605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:25.243605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:25.243605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:25.243605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:25.243605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:18:25.243605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:18:25.243605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:18:25.243605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:18:25.243605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:25.243605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:25.243605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:25.243605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:25.243605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:25.243605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:25.245290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:25.245290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:25.245290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:25.245290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:25.245290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:25.245290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:25.245290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:25.245290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:25.245290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:25.245290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:25.245290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:18:25.245290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:18:25.245290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 21:18:25.245290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:25.250543 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 388/766] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:18:24 2022] GET /identity/v3/services/network => generated 88 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:25.253426 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a0498-3aad-4777-861a-ec6c9873f9ac None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:25.254339 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a0498-3aad-4777-861a-ec6c9873f9ac None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:25.255198 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a0498-3aad-4777-861a-ec6c9873f9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:25.287357 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-257a0498-3aad-4777-861a-ec6c9873f9ac None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:25.289966 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-257a0498-3aad-4777-861a-ec6c9873f9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:25.293962 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:25.369065 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:25.393117 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 389/767] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:18:25.394056 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257a0498-3aad-4777-861a-ec6c9873f9ac None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:25.424009 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e68e9e5-9dab-456c-a8cd-ab53f6242b24 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:25.496582 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032b1459-e6d7-410c-aefb-7f1986f0c76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:25.498774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b1459-e6d7-410c-aefb-7f1986f0c76d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:25.499661 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b1459-e6d7-410c-aefb-7f1986f0c76d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:25.500528 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b1459-e6d7-410c-aefb-7f1986f0c76d None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:25.502732 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032b1459-e6d7-410c-aefb-7f1986f0c76d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:25.503762 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032b1459-e6d7-410c-aefb-7f1986f0c76d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:25.507870 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032b1459-e6d7-410c-aefb-7f1986f0c76d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:25.520517 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257a0498-3aad-4777-861a-ec6c9873f9ac None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:25.535797 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 379/768] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:25.538116 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032b1459-e6d7-410c-aefb-7f1986f0c76d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:25.564421 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a3cc2f-7981-4379-8e96-5e68cc9542be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:25.565129 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3cc2f-7981-4379-8e96-5e68cc9542be None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:25.565420 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3cc2f-7981-4379-8e96-5e68cc9542be None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:25.565680 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3cc2f-7981-4379-8e96-5e68cc9542be None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:25.566431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a3cc2f-7981-4379-8e96-5e68cc9542be None admin] RBAC: Adding query filter params (name=network) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:25.566749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a3cc2f-7981-4379-8e96-5e68cc9542be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:25.568529 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a3cc2f-7981-4379-8e96-5e68cc9542be None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:25.575195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 390/769] 10.222.0.65 () {56 vars in 1112 bytes} [Thu May 19 21:18:25 2022] GET /identity/v3/services?name=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:25.576373 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a3cc2f-7981-4379-8e96-5e68cc9542be None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:25.640656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:25.706589 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a3cc2f-7981-4379-8e96-5e68cc9542be None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:25.737948 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:25.747361 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032b1459-e6d7-410c-aefb-7f1986f0c76d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:25.751871 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b21c42-2991-4376-8cf1-681e1f590bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:25.752718 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b21c42-2991-4376-8cf1-681e1f590bff None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:25.753010 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b21c42-2991-4376-8cf1-681e1f590bff None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:25.753281 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b21c42-2991-4376-8cf1-681e1f590bff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:25.766296 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b21c42-2991-4376-8cf1-681e1f590bff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:25.767750 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b21c42-2991-4376-8cf1-681e1f590bff None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:25.785537 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-32b21c42-2991-4376-8cf1-681e1f590bff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:25.785537 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:25.785537 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:25.785537 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:25.785537 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:25.785537 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:25.785537 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:25.785537 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:25.785537 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:25.785537 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:25.785537 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:25.785537 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:25.785537 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:18:25.785537 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:18:25.785537 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:18:25.787693 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:18:25.787693 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:25.787693 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:25.787693 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:25.787693 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:25.787693 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:25.787693 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:25.787693 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:25.787693 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:25.787693 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:25.787693 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:25.787693 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:25.787693 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:25.787693 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:25.787693 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:25.787693 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:25.787693 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:25.789369 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:18:25.789369 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:18:25.789369 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:18:25.789369 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:18:25.789369 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:18:25.789369 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:18:25.789369 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:25.789369 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:25.789369 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 391/770] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:18:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:25.789369 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b21c42-2991-4376-8cf1-681e1f590bff None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:25.820971 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0906c5-1286-4291-802f-881d99b8a74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:25.823263 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0906c5-1286-4291-802f-881d99b8a74b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:25.824148 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0906c5-1286-4291-802f-881d99b8a74b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:25.825053 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0906c5-1286-4291-802f-881d99b8a74b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:25.826949 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0906c5-1286-4291-802f-881d99b8a74b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:25.830032 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0906c5-1286-4291-802f-881d99b8a74b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:25.856706 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-0b0906c5-1286-4291-802f-881d99b8a74b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 21:18:25.856706 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:25.856706 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:18:25.856706 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:18:25.856706 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:18:25.856706 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:18:25.856706 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:18:25.856706 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:18:25.856706 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 21:18:25.856706 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:25.856706 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:18:25.856706 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:25.856706 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:25.856706 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:25.856706 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:25.856706 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:25.858964 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:25.858964 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:25.858964 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:25.858964 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:25.858964 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:25.858964 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:25.858964 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:25.858964 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:18:25.858964 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:18:25.858964 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:18:25.858964 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:18:25.858964 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:25.858964 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:25.858964 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:25.858964 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:25.858964 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:25.860595 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:25.860595 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:25.860595 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:25.860595 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:25.860595 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:25.860595 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:25.860595 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:25.860595 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:25.860595 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:25.860595 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:25.860595 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:25.860595 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:18:25.860595 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:18:25.860595 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 21:18:25.860595 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:25.865967 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 380/771] 10.222.0.65 () {56 vars in 1129 bytes} [Thu May 19 21:18:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:25.868010 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0906c5-1286-4291-802f-881d99b8a74b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:25.930877 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b21c42-2991-4376-8cf1-681e1f590bff None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:25.950051 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:25.999138 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e6500d-5a1d-42cd-976a-c06d2387afd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:26.000790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6500d-5a1d-42cd-976a-c06d2387afd2 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:26.001599 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6500d-5a1d-42cd-976a-c06d2387afd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:26.002454 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6500d-5a1d-42cd-976a-c06d2387afd2 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:26.004124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e6500d-5a1d-42cd-976a-c06d2387afd2 None admin] RBAC: Adding query filter params (type=network) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:26.004982 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e6500d-5a1d-42cd-976a-c06d2387afd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:26.008046 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e6500d-5a1d-42cd-976a-c06d2387afd2 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:26.018311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 392/772] 10.222.0.65 () {56 vars in 1112 bytes} [Thu May 19 21:18:25 2022] GET /identity/v3/services?type=network => generated 350 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:26.019891 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0906c5-1286-4291-802f-881d99b8a74b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:26.020436 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e6500d-5a1d-42cd-976a-c06d2387afd2 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:26.062943 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a404f1-d7af-4579-a6a0-ca5afb961d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:26.063620 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a404f1-d7af-4579-a6a0-ca5afb961d8f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:26.063938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a404f1-d7af-4579-a6a0-ca5afb961d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:26.064234 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a404f1-d7af-4579-a6a0-ca5afb961d8f None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:26.065805 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a404f1-d7af-4579-a6a0-ca5afb961d8f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:26.066138 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a404f1-d7af-4579-a6a0-ca5afb961d8f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:26.067709 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a404f1-d7af-4579-a6a0-ca5afb961d8f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:26.086644 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 381/773] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 21:18:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:26.087699 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a404f1-d7af-4579-a6a0-ca5afb961d8f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:26.146739 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:26.205578 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e6500d-5a1d-42cd-976a-c06d2387afd2 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:26.246397 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ce64cf-cddc-4806-ab2c-c4b7b6d9a13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:26.247555 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce64cf-cddc-4806-ab2c-c4b7b6d9a13d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:26.248042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce64cf-cddc-4806-ab2c-c4b7b6d9a13d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:26.248541 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce64cf-cddc-4806-ab2c-c4b7b6d9a13d None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:26.249821 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ce64cf-cddc-4806-ab2c-c4b7b6d9a13d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:26.250457 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ce64cf-cddc-4806-ab2c-c4b7b6d9a13d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:26.253074 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ce64cf-cddc-4806-ab2c-c4b7b6d9a13d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:26.266692 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 393/774] 10.222.0.65 () {56 vars in 1124 bytes} [Thu May 19 21:18:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:26.266917 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a404f1-d7af-4579-a6a0-ca5afb961d8f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:26.267017 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ce64cf-cddc-4806-ab2c-c4b7b6d9a13d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:26.314795 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cd7b9-eabd-4e2c-8ebb-30d7ec874244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:26.315474 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cd7b9-eabd-4e2c-8ebb-30d7ec874244 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:26.315749 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cd7b9-eabd-4e2c-8ebb-30d7ec874244 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:26.316021 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cd7b9-eabd-4e2c-8ebb-30d7ec874244 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:26.316824 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9cd7b9-eabd-4e2c-8ebb-30d7ec874244 None admin] RBAC: Adding query filter params (interface=public, service_id=105d03df6a5a4f94b6e414a6b3b223d6, region_id=RegionOne) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:26.317132 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9cd7b9-eabd-4e2c-8ebb-30d7ec874244 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:26.318180 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:26.318610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9cd7b9-eabd-4e2c-8ebb-30d7ec874244 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:26.327264 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 382/775] 10.222.0.65 () {56 vars in 1251 bytes} [Thu May 19 21:18:26 2022] GET /identity/v3/endpoints?service_id=105d03df6a5a4f94b6e414a6b3b223d6&interface=public®ion_id=RegionOne => generated 194 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:26.328677 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342bdb8-0052-467f-8415-e6da16ebab15 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:26.328966 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342bdb8-0052-467f-8415-e6da16ebab15 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:26.329238 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342bdb8-0052-467f-8415-e6da16ebab15 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:26.330755 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 383/776] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:18:26.335957 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7342bdb8-0052-467f-8415-e6da16ebab15 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:26.410465 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ce64cf-cddc-4806-ab2c-c4b7b6d9a13d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:26.465404 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710981d5-6db9-48f6-8f0f-78b2b6508973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:26.467066 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710981d5-6db9-48f6-8f0f-78b2b6508973 None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:26.467564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710981d5-6db9-48f6-8f0f-78b2b6508973 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:26.468054 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710981d5-6db9-48f6-8f0f-78b2b6508973 None admin] PATH_INFO: `/v3/projects/c4940befa9c4405699c72deedc8a2127/users/e254671b21fe4661a381b9521810cf80/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:26.468887 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:26.471675 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710981d5-6db9-48f6-8f0f-78b2b6508973 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4940befa9c4405699c72deedc8a2127, user_id=e254671b21fe4661a381b9521810cf80, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:26.474494 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710981d5-6db9-48f6-8f0f-78b2b6508973 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:26.492472 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7342bdb8-0052-467f-8415-e6da16ebab15 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:26.502725 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 394/777] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 21:18:26 2022] PUT /identity/v3/projects/c4940befa9c4405699c72deedc8a2127/users/e254671b21fe4661a381b9521810cf80/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:18:26.502725 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710981d5-6db9-48f6-8f0f-78b2b6508973 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:26.532640 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759e7a17-3b73-43a7-94b2-580aa8a1075a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:26.533445 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e7a17-3b73-43a7-94b2-580aa8a1075a None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:26.533788 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e7a17-3b73-43a7-94b2-580aa8a1075a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:26.534128 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e7a17-3b73-43a7-94b2-580aa8a1075a None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:26.535007 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759e7a17-3b73-43a7-94b2-580aa8a1075a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:26.535380 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759e7a17-3b73-43a7-94b2-580aa8a1075a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:26.536882 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759e7a17-3b73-43a7-94b2-580aa8a1075a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:26.547851 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 384/778] 10.222.0.65 () {56 vars in 1124 bytes} [Thu May 19 21:18:26 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:26.547851 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171ed6d-7077-4082-9598-5baa07bc6312 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:26.547851 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171ed6d-7077-4082-9598-5baa07bc6312 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:26.547851 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171ed6d-7077-4082-9598-5baa07bc6312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:26.566060 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-9171ed6d-7077-4082-9598-5baa07bc6312 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:26.567521 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9171ed6d-7077-4082-9598-5baa07bc6312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:26.663020 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 385/779] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:18:26.665497 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171ed6d-7077-4082-9598-5baa07bc6312 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:26.674709 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:26.684417 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710981d5-6db9-48f6-8f0f-78b2b6508973 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:26.729164 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a76b97-a312-4fb1-9007-89a821fe5502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:26.731127 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a76b97-a312-4fb1-9007-89a821fe5502 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:26.731706 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a76b97-a312-4fb1-9007-89a821fe5502 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:26.732201 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a76b97-a312-4fb1-9007-89a821fe5502 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:26.733746 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a76b97-a312-4fb1-9007-89a821fe5502 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:26.736577 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a76b97-a312-4fb1-9007-89a821fe5502 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:26.746201 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 395/780] 10.222.0.65 () {56 vars in 1087 bytes} [Thu May 19 21:18:26 2022] GET /identity/v3/services => generated 772 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:26.747619 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a76b97-a312-4fb1-9007-89a821fe5502 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:26.835893 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:26.883701 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171ed6d-7077-4082-9598-5baa07bc6312 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:26.929300 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6838e-fbed-4058-89e8-166257522e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:26.930215 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6838e-fbed-4058-89e8-166257522e17 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:26.930619 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6838e-fbed-4058-89e8-166257522e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:26.930991 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6838e-fbed-4058-89e8-166257522e17 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:26.931957 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b6838e-fbed-4058-89e8-166257522e17 None admin] RBAC: Adding query filter params (interface=public, service_id=889198e915d9491b8b2e636ea7ba0275, region_id=RegionOne) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:26.932375 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b6838e-fbed-4058-89e8-166257522e17 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:26.933941 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b6838e-fbed-4058-89e8-166257522e17 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:26.940895 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 386/781] 10.222.0.65 () {56 vars in 1251 bytes} [Thu May 19 21:18:26 2022] GET /identity/v3/endpoints?service_id=889198e915d9491b8b2e636ea7ba0275&interface=public®ion_id=RegionOne => generated 194 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:26.947327 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6838e-fbed-4058-89e8-166257522e17 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:27.013762 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a76b97-a312-4fb1-9007-89a821fe5502 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:27.061513 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe235c9-665a-4953-8365-35a4032b5249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:27.063146 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe235c9-665a-4953-8365-35a4032b5249 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:27.063802 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe235c9-665a-4953-8365-35a4032b5249 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:27.064370 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe235c9-665a-4953-8365-35a4032b5249 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:27.066830 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6838e-fbed-4058-89e8-166257522e17 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:27.078467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe235c9-665a-4953-8365-35a4032b5249 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:18:27.088428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-bfe235c9-665a-4953-8365-35a4032b5249 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:18:27.088428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:27.088428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:27.088428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:27.088428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:27.088428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:27.088428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:27.088428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:27.088428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:27.088428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:27.088428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:27.088428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:27.088428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:18:27.088428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:18:27.088428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:18:27.088428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:18:27.089431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:18:27.089431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:18:27.089431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:18:27.089431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:18:27.089431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:27.089431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:27.089431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:27.089431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:27.089431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:27.089431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:27.089431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:27.089431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:27.089431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:27.089431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:27.089431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:27.089431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:27.090314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:27.090314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:27.090314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:27.090314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:27.090314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:18:27.090314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:18:27.090314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:18:27.090314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:18:27.090314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:18:27.090314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:18:27.090314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:18:27.090314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:27.093362 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 396/782] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 21:18:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:27.101890 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe235c9-665a-4953-8365-35a4032b5249 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:27.130103 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892098ce-2860-4859-a3fc-a9c642635c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:27.132001 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892098ce-2860-4859-a3fc-a9c642635c5d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:27.132886 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892098ce-2860-4859-a3fc-a9c642635c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:27.133375 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892098ce-2860-4859-a3fc-a9c642635c5d None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:27.134737 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892098ce-2860-4859-a3fc-a9c642635c5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:27.136922 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892098ce-2860-4859-a3fc-a9c642635c5d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:27.150860 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 387/783] 10.222.0.65 () {56 vars in 1087 bytes} [Thu May 19 21:18:26 2022] GET /identity/v3/services => generated 772 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:18:27.156037 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:27.303722 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe235c9-665a-4953-8365-35a4032b5249 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:27.356411 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77506e2a-7502-46a8-a382-3cfaf952abef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:27.357508 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77506e2a-7502-46a8-a382-3cfaf952abef None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:27.358106 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77506e2a-7502-46a8-a382-3cfaf952abef None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:27.358654 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77506e2a-7502-46a8-a382-3cfaf952abef None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:27.359699 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77506e2a-7502-46a8-a382-3cfaf952abef None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:27.360238 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77506e2a-7502-46a8-a382-3cfaf952abef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:27.361905 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77506e2a-7502-46a8-a382-3cfaf952abef None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:27.376782 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 397/784] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 21:18:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:18:27.383872 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892098ce-2860-4859-a3fc-a9c642635c5d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:27.399102 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:27.472302 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8babd8-aa04-48db-93ea-767928e87c2b None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:27.472737 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8babd8-aa04-48db-93ea-767928e87c2b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:27.472893 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8babd8-aa04-48db-93ea-767928e87c2b None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:27.474944 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 398/785] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:18:27.482817 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ead8a-026c-41f7-862c-9227677244ea None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:27.483317 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ead8a-026c-41f7-862c-9227677244ea None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:27.483809 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ead8a-026c-41f7-862c-9227677244ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:27.508952 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-992ead8a-026c-41f7-862c-9227677244ea None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:27.515424 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-992ead8a-026c-41f7-862c-9227677244ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:27.535480 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892098ce-2860-4859-a3fc-a9c642635c5d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:27.582601 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe308e2b-75b3-431f-9767-a29e2f777926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:27.583492 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe308e2b-75b3-431f-9767-a29e2f777926 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:27.583951 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe308e2b-75b3-431f-9767-a29e2f777926 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:27.585505 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe308e2b-75b3-431f-9767-a29e2f777926 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:27.603776 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe308e2b-75b3-431f-9767-a29e2f777926 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:27.605517 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe308e2b-75b3-431f-9767-a29e2f777926 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:27.637290 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-fe308e2b-75b3-431f-9767-a29e2f777926 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:27.637290 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:27.637290 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:27.637290 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:27.637290 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:27.637290 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:27.637290 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:27.637290 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:27.637290 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:27.637290 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:27.637290 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:27.637290 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:27.637290 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:27.637290 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:27.637290 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:27.638226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:27.638226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:27.638226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:27.638226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:27.638226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:27.638226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:27.638226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:27.638226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:27.638226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:27.638226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:27.638226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:27.638226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:27.638226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:27.638226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:27.638226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:27.638226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:27.638226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:27.639184 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:27.639184 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:27.639184 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:27.639184 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:27.646211 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 388/786] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:27.648274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe308e2b-75b3-431f-9767-a29e2f777926 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:27.685194 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 399/787] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:18:27.715024 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ead8a-026c-41f7-862c-9227677244ea None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:27.752596 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:27.753905 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:27.828832 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe308e2b-75b3-431f-9767-a29e2f777926 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:27.906957 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dda533-63f9-49a6-8821-a7938b88449c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:27.907682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dda533-63f9-49a6-8821-a7938b88449c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:27.908028 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dda533-63f9-49a6-8821-a7938b88449c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:27.908371 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dda533-63f9-49a6-8821-a7938b88449c None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:27.909177 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dda533-63f9-49a6-8821-a7938b88449c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:27.909545 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dda533-63f9-49a6-8821-a7938b88449c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:27.911307 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dda533-63f9-49a6-8821-a7938b88449c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:27.931069 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 389/788] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:27.932558 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dda533-63f9-49a6-8821-a7938b88449c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:27.952630 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ead8a-026c-41f7-862c-9227677244ea None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:28.041009 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f260a00-ca49-4e13-9434-fca9e4dac6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:28.042287 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f260a00-ca49-4e13-9434-fca9e4dac6d8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:28.042775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f260a00-ca49-4e13-9434-fca9e4dac6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:28.043250 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f260a00-ca49-4e13-9434-fca9e4dac6d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:28.077725 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f260a00-ca49-4e13-9434-fca9e4dac6d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:18:28.086302 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-7f260a00-ca49-4e13-9434-fca9e4dac6d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:18:28.086302 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:28.086302 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:28.086302 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:28.086302 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:28.086302 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:28.086302 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:28.086302 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:28.086302 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:28.086302 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:28.086302 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:28.086302 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:28.086302 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:18:28.086302 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:18:28.086302 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:18:28.086302 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:18:28.087304 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:18:28.087304 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:18:28.087304 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:18:28.087304 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:18:28.087304 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:28.087304 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:28.087304 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:28.087304 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:28.087304 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:28.087304 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:28.087304 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:28.087304 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:28.087304 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:28.087304 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:28.087304 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:28.087304 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:28.088136 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:28.088136 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:28.088136 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:28.088136 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:28.088136 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:18:28.088136 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:18:28.088136 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:18:28.088136 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:18:28.088136 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:18:28.088136 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:18:28.088136 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:18:28.088136 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:28.091024 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 400/789] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 21:18:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:28.097126 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f260a00-ca49-4e13-9434-fca9e4dac6d8 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:28.110287 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dda533-63f9-49a6-8821-a7938b88449c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:28.176567 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:28.230074 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aee280-a817-4bdd-b3c9-b67205dbfe56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:28.231496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aee280-a817-4bdd-b3c9-b67205dbfe56 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:28.231983 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aee280-a817-4bdd-b3c9-b67205dbfe56 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:28.232477 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aee280-a817-4bdd-b3c9-b67205dbfe56 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:28.276900 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aee280-a817-4bdd-b3c9-b67205dbfe56 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:28.278434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aee280-a817-4bdd-b3c9-b67205dbfe56 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:28.303166 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-27aee280-a817-4bdd-b3c9-b67205dbfe56 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 21:18:28.303166 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:28.303166 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:28.303166 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:28.303166 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:28.303166 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:28.303166 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:28.303166 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:28.303166 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:28.303166 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:28.303166 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:28.303166 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:28.303166 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:18:28.303166 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:18:28.303166 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:18:28.303166 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:18:28.304182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:28.304182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:28.304182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:18:28.304182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:28.304182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:18:28.304182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:28.304182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:28.304182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:28.304182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:28.304182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:28.304182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:28.304182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:28.304182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:28.304182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:28.304182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:28.304182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:28.304182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:28.304182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:28.305127 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:28.305127 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:28.305127 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:18:28.305127 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:18:28.305127 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:18:28.305127 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:18:28.305127 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:18:28.305127 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:18:28.305127 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 21:18:28.305127 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:28.305127 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 390/790] 10.222.0.65 () {56 vars in 1117 bytes} [Thu May 19 21:18:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:28.315370 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aee280-a817-4bdd-b3c9-b67205dbfe56 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:28.336032 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:28.391569 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f260a00-ca49-4e13-9434-fca9e4dac6d8 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:28.502314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d4388-624d-4615-8464-b8d990991ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:28.503393 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d4388-624d-4615-8464-b8d990991ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:28.503888 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d4388-624d-4615-8464-b8d990991ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:28.504382 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d4388-624d-4615-8464-b8d990991ab9 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:28.505807 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2d4388-624d-4615-8464-b8d990991ab9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:28.506396 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2d4388-624d-4615-8464-b8d990991ab9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:28.508994 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2d4388-624d-4615-8464-b8d990991ab9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:28.552389 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 401/791] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:18:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 456 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:28.559644 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d4388-624d-4615-8464-b8d990991ab9 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:28.608355 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aee280-a817-4bdd-b3c9-b67205dbfe56 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:28.717467 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23cd3f6-c6c2-4498-9e44-9aa66962cb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:28.726295 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cd3f6-c6c2-4498-9e44-9aa66962cb43 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:28.726295 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cd3f6-c6c2-4498-9e44-9aa66962cb43 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:28.726295 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cd3f6-c6c2-4498-9e44-9aa66962cb43 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:28.726295 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23cd3f6-c6c2-4498-9e44-9aa66962cb43 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:28.726295 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23cd3f6-c6c2-4498-9e44-9aa66962cb43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:28.726295 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23cd3f6-c6c2-4498-9e44-9aa66962cb43 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:28.770151 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 391/792] 10.222.0.65 () {56 vars in 1149 bytes} [Thu May 19 21:18:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 452 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:28.778211 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23cd3f6-c6c2-4498-9e44-9aa66962cb43 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:28.796627 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d4388-624d-4615-8464-b8d990991ab9 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:28.843636 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9084f8f2-910e-4140-8bfc-88604bb0242a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:28.845209 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084f8f2-910e-4140-8bfc-88604bb0242a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:28.845704 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084f8f2-910e-4140-8bfc-88604bb0242a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:28.846180 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084f8f2-910e-4140-8bfc-88604bb0242a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:28.860358 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:28.876847 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9084f8f2-910e-4140-8bfc-88604bb0242a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:28.879784 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9084f8f2-910e-4140-8bfc-88604bb0242a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:28.886022 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:28.908517 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-9084f8f2-910e-4140-8bfc-88604bb0242a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:28.908517 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:28.908517 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:28.908517 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:28.908517 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:28.908517 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:28.908517 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:28.908517 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:28.908517 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:28.908517 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:28.908517 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:28.908517 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:28.908517 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:28.908517 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:28.908517 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:28.914757 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:28.914757 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:28.914757 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:28.914757 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:28.914757 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:28.914757 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:28.914757 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:28.914757 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:28.914757 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:28.914757 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:28.914757 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:28.914757 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:28.914757 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:28.914757 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:28.914757 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:28.914757 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:28.914757 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:28.916745 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:28.916745 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:28.916745 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:28.916745 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:28.916745 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 402/793] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:28.916745 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9084f8f2-910e-4140-8bfc-88604bb0242a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:29.091782 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23cd3f6-c6c2-4498-9e44-9aa66962cb43 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:29.126547 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9084f8f2-910e-4140-8bfc-88604bb0242a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:29.160621 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa2aee5-4179-437a-90db-556be66fdef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:29.162810 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2aee5-4179-437a-90db-556be66fdef7 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:29.163621 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2aee5-4179-437a-90db-556be66fdef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:29.164389 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2aee5-4179-437a-90db-556be66fdef7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:29.185250 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa2aee5-4179-437a-90db-556be66fdef7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:29.188379 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa2aee5-4179-437a-90db-556be66fdef7 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:29.225436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-6fa2aee5-4179-437a-90db-556be66fdef7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:29.225436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:29.225436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:29.225436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:29.225436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:29.225436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:29.225436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:29.225436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:29.225436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:29.225436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:29.225436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:29.225436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:29.225436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:29.225436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:29.225436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:29.227502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:29.227502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:29.227502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:29.227502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:29.227502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:29.227502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:29.227502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:29.227502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:29.227502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:29.227502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:29.227502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:29.227502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:29.227502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:29.227502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:29.227502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:29.227502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:29.227502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:29.229366 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:29.229366 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:29.229366 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:29.229366 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:29.229797 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8794641e-f367-4c84-a9a9-24145408eb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:29.229797 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794641e-f367-4c84-a9a9-24145408eb91 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:29.229797 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794641e-f367-4c84-a9a9-24145408eb91 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:29.229797 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794641e-f367-4c84-a9a9-24145408eb91 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:29.230336 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8794641e-f367-4c84-a9a9-24145408eb91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:29.230336 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8794641e-f367-4c84-a9a9-24145408eb91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:29.230336 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8794641e-f367-4c84-a9a9-24145408eb91 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:29.238584 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 392/794] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 457 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:29.244295 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa2aee5-4179-437a-90db-556be66fdef7 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:29.257174 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 403/795] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:29.263458 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8794641e-f367-4c84-a9a9-24145408eb91 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:29.372257 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:29.435021 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:29.461681 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa2aee5-4179-437a-90db-556be66fdef7 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:29.528166 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8794641e-f367-4c84-a9a9-24145408eb91 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:29.534858 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94ed3b1-b9b9-4d2d-92e6-abaea72306e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:29.538092 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ed3b1-b9b9-4d2d-92e6-abaea72306e1 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:29.538927 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ed3b1-b9b9-4d2d-92e6-abaea72306e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:29.539730 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ed3b1-b9b9-4d2d-92e6-abaea72306e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:29.541409 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94ed3b1-b9b9-4d2d-92e6-abaea72306e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:29.542278 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94ed3b1-b9b9-4d2d-92e6-abaea72306e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:29.546036 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94ed3b1-b9b9-4d2d-92e6-abaea72306e1 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:29.578574 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 393/796] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:29.583655 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94ed3b1-b9b9-4d2d-92e6-abaea72306e1 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:29.624556 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48170cc-b452-4aa1-9d7b-8240c9b91ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:29.625783 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48170cc-b452-4aa1-9d7b-8240c9b91ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:29.626224 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48170cc-b452-4aa1-9d7b-8240c9b91ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:29.626718 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48170cc-b452-4aa1-9d7b-8240c9b91ec5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:29.640250 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48170cc-b452-4aa1-9d7b-8240c9b91ec5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:29.641769 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48170cc-b452-4aa1-9d7b-8240c9b91ec5 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:29.669482 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-c48170cc-b452-4aa1-9d7b-8240c9b91ec5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 21:18:29.669482 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:29.669482 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:29.669482 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:29.669482 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:29.669482 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:29.669482 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:29.669482 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:29.669482 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:29.669482 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:29.669482 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:29.669482 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:29.669482 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:18:29.669482 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:18:29.669482 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:18:29.669482 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:18:29.671374 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:29.671374 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:29.671374 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:18:29.671374 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:29.671374 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:18:29.671374 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:29.671374 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:29.671374 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:29.671374 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:29.671374 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:29.671374 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:29.671374 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:29.671374 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:29.671374 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:29.671374 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:29.671374 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:29.671374 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:29.671374 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:29.673819 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:29.673819 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:29.673819 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:18:29.673819 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:18:29.673819 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:18:29.673819 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:18:29.673819 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:18:29.673819 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:18:29.673819 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 21:18:29.673819 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:29.673819 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 404/797] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 21:18:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 409 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:18:29.674959 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa705f-b222-4337-ab46-51756ed0c0be None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:29.674959 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa705f-b222-4337-ab46-51756ed0c0be None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:29.675587 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa705f-b222-4337-ab46-51756ed0c0be None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:29.682258 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 405/798] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:18:29.683695 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa705f-b222-4337-ab46-51756ed0c0be None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:29.808452 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94ed3b1-b9b9-4d2d-92e6-abaea72306e1 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:29.847095 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa705f-b222-4337-ab46-51756ed0c0be None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:29.855092 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:29.888069 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d54e98c-ebdc-45c9-a569-7ffc5df65f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:29.888727 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54e98c-ebdc-45c9-a569-7ffc5df65f6a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:29.888964 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54e98c-ebdc-45c9-a569-7ffc5df65f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:29.889243 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54e98c-ebdc-45c9-a569-7ffc5df65f6a None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:29.890108 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d54e98c-ebdc-45c9-a569-7ffc5df65f6a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:29.890458 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d54e98c-ebdc-45c9-a569-7ffc5df65f6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:29.891946 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d54e98c-ebdc-45c9-a569-7ffc5df65f6a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:29.893054 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0dfb33-875f-4083-86db-af032f614247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:29.895171 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0dfb33-875f-4083-86db-af032f614247 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:29.895790 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0dfb33-875f-4083-86db-af032f614247 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:29.896421 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0dfb33-875f-4083-86db-af032f614247 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:29.909930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0dfb33-875f-4083-86db-af032f614247 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:29.914381 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 406/799] 10.222.0.65 () {56 vars in 1135 bytes} [Thu May 19 21:18:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:29.914628 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0dfb33-875f-4083-86db-af032f614247 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:29.914792 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192adba-dd80-4e37-9f96-86b1214a1de5 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:29.915301 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192adba-dd80-4e37-9f96-86b1214a1de5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:29.915796 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192adba-dd80-4e37-9f96-86b1214a1de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:29.935321 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-3192adba-dd80-4e37-9f96-86b1214a1de5 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:29.936538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3192adba-dd80-4e37-9f96-86b1214a1de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:29.948133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-3e0dfb33-875f-4083-86db-af032f614247 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:29.948133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:29.948133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:29.948133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:29.948133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:29.948133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:29.948133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:29.948133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:29.948133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:29.948133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:29.948133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:29.948133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:29.948133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:18:29.948133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:18:29.948133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:18:29.949282 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:18:29.949282 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:29.949282 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:29.949282 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:29.949282 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:29.949282 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:29.949282 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:29.949282 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:29.949282 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:29.949282 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:29.949282 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:29.949282 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:29.949282 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:29.949282 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:29.949282 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:29.949282 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:29.949282 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:29.951019 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:18:29.951019 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:18:29.951019 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:18:29.951019 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:18:29.951019 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:18:29.951019 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:18:29.951019 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:29.951019 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:29.957828 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 394/800] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:18:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:29.960904 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5bb4de-c02d-4e1b-8ba8-3835c45bc956 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:29.961405 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5bb4de-c02d-4e1b-8ba8-3835c45bc956 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:29.961909 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5bb4de-c02d-4e1b-8ba8-3835c45bc956 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:29.964820 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 395/801] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:18:29.966318 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5bb4de-c02d-4e1b-8ba8-3835c45bc956 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:29.986136 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:30.051400 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 407/802] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:18:30.052483 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192adba-dd80-4e37-9f96-86b1214a1de5 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:30.132168 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5bb4de-c02d-4e1b-8ba8-3835c45bc956 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:30.177504 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5133545d-42a1-4995-8f1d-4e62c431ac57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:30.178345 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133545d-42a1-4995-8f1d-4e62c431ac57 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:30.178635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133545d-42a1-4995-8f1d-4e62c431ac57 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:30.178912 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133545d-42a1-4995-8f1d-4e62c431ac57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:30.193685 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5133545d-42a1-4995-8f1d-4e62c431ac57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:30.194963 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5133545d-42a1-4995-8f1d-4e62c431ac57 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:30.207212 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:30.216100 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-5133545d-42a1-4995-8f1d-4e62c431ac57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:30.216100 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:30.216100 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:30.216100 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:30.216100 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:30.216100 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:30.216100 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:30.216100 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:30.216100 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:30.216100 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:30.216100 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:30.216100 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:30.216100 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:30.216100 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:30.216100 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:30.217098 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:30.217098 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:30.217098 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:30.217098 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:30.217098 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:30.217098 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:30.217098 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:30.217098 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:30.217098 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:30.217098 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:30.217098 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:30.217098 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:30.217098 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:30.217098 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:30.217098 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:30.217098 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:30.217098 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:30.217980 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:30.217980 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:30.217980 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:30.217980 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:30.217980 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 396/803] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:30.218755 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ba93d-f983-44bf-b7eb-afd4409136a0 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:30.219120 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ba93d-f983-44bf-b7eb-afd4409136a0 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:30.219403 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ba93d-f983-44bf-b7eb-afd4409136a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:30.219699 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192adba-dd80-4e37-9f96-86b1214a1de5 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:30.246102 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-010ba93d-f983-44bf-b7eb-afd4409136a0 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:30.248278 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-010ba93d-f983-44bf-b7eb-afd4409136a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:30.264970 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c52ca2-1bf8-4459-ae9a-3238936f2217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:30.265655 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c52ca2-1bf8-4459-ae9a-3238936f2217 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:30.265944 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c52ca2-1bf8-4459-ae9a-3238936f2217 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:30.266214 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c52ca2-1bf8-4459-ae9a-3238936f2217 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:30.267156 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c52ca2-1bf8-4459-ae9a-3238936f2217 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:30.267467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c52ca2-1bf8-4459-ae9a-3238936f2217 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:30.269017 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c52ca2-1bf8-4459-ae9a-3238936f2217 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:30.286759 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 408/804] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 21:18:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:30.286759 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c52ca2-1bf8-4459-ae9a-3238936f2217 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:30.335094 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:30.358293 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 397/805] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:18:30.358293 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ba93d-f983-44bf-b7eb-afd4409136a0 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:30.438477 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c52ca2-1bf8-4459-ae9a-3238936f2217 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:30.479479 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c4cdf1-f538-4cbc-96ee-869e637d7789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:30.480343 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c4cdf1-f538-4cbc-96ee-869e637d7789 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:30.480643 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c4cdf1-f538-4cbc-96ee-869e637d7789 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:30.480902 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c4cdf1-f538-4cbc-96ee-869e637d7789 None admin] PATH_INFO: `/v3/services/network` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:30.481627 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c4cdf1-f538-4cbc-96ee-869e637d7789 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:30.483164 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c4cdf1-f538-4cbc-96ee-869e637d7789 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:30.497698 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-40c4cdf1-f538-4cbc-96ee-869e637d7789 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 21:18:30.497698 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:30.497698 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:18:30.497698 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:18:30.497698 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:18:30.497698 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:18:30.497698 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:18:30.497698 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:18:30.497698 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 21:18:30.497698 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:30.497698 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:18:30.497698 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:30.497698 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:30.497698 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:30.497698 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:30.497698 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:30.497698 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:30.499082 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:30.499082 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:30.499082 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:30.499082 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:30.499082 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:30.499082 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:30.499082 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:18:30.499082 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:18:30.499082 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:18:30.499082 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:18:30.499082 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:30.499082 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:30.499082 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:30.499082 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:30.499082 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:30.499082 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:30.500659 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:30.500659 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:30.500659 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:30.500659 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:30.500659 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:30.500659 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:30.500659 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:30.500659 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:30.500659 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:30.500659 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:30.500659 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:18:30.500659 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:18:30.500659 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 21:18:30.500659 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:30.500659 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 409/806] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:18:30 2022] GET /identity/v3/services/network => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:30.502201 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c4cdf1-f538-4cbc-96ee-869e637d7789 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:30.529453 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:30.614018 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ba93d-f983-44bf-b7eb-afd4409136a0 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:30.621474 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c4cdf1-f538-4cbc-96ee-869e637d7789 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:30.652764 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725dbcd7-cf20-48f8-bd38-8d70dd518fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:30.653830 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725dbcd7-cf20-48f8-bd38-8d70dd518fef None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:30.654320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725dbcd7-cf20-48f8-bd38-8d70dd518fef None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:30.654717 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725dbcd7-cf20-48f8-bd38-8d70dd518fef None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:30.655972 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725dbcd7-cf20-48f8-bd38-8d70dd518fef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:30.656478 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725dbcd7-cf20-48f8-bd38-8d70dd518fef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:30.658904 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725dbcd7-cf20-48f8-bd38-8d70dd518fef None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:30.661037 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526eb1b0-13ee-417c-a631-8093641da287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:30.662474 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526eb1b0-13ee-417c-a631-8093641da287 None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:30.663103 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526eb1b0-13ee-417c-a631-8093641da287 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:30.663427 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526eb1b0-13ee-417c-a631-8093641da287 None admin] PATH_INFO: `/v3/projects/c4940befa9c4405699c72deedc8a2127/users/eba91dff7cac401e81f6fb6391874af1/roles/9dc26e65165b49d6916b54458f847755` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:30.677265 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 398/807] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:30.680913 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2cb51-ecfd-4187-8e36-084abfe509a6 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:30.681406 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2cb51-ecfd-4187-8e36-084abfe509a6 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:30.681887 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2cb51-ecfd-4187-8e36-084abfe509a6 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:30.685037 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 399/808] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:18:30.685545 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b2cb51-ecfd-4187-8e36-084abfe509a6 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:30.696004 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526eb1b0-13ee-417c-a631-8093641da287 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4940befa9c4405699c72deedc8a2127, user_id=eba91dff7cac401e81f6fb6391874af1, role_id=9dc26e65165b49d6916b54458f847755)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:30.698452 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526eb1b0-13ee-417c-a631-8093641da287 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:30.715873 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 410/809] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 21:18:30 2022] PUT /identity/v3/projects/c4940befa9c4405699c72deedc8a2127/users/eba91dff7cac401e81f6fb6391874af1/roles/9dc26e65165b49d6916b54458f847755 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:18:30.717014 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526eb1b0-13ee-417c-a631-8093641da287 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:30.777503 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:30.845727 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:30.876345 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b2cb51-ecfd-4187-8e36-084abfe509a6 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:30.901824 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526eb1b0-13ee-417c-a631-8093641da287 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:30.938206 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e96a5-feae-4cf8-9645-e4340b0a6654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:30.939526 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e96a5-feae-4cf8-9645-e4340b0a6654 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:30.939991 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e96a5-feae-4cf8-9645-e4340b0a6654 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:30.940543 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e96a5-feae-4cf8-9645-e4340b0a6654 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:30.958697 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616e96a5-feae-4cf8-9645-e4340b0a6654 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:18:30.967795 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca9789-db11-45eb-bc7c-f729555bd821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:30.969135 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca9789-db11-45eb-bc7c-f729555bd821 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:30.969789 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca9789-db11-45eb-bc7c-f729555bd821 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:30.970479 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca9789-db11-45eb-bc7c-f729555bd821 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:30.971935 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ca9789-db11-45eb-bc7c-f729555bd821 None admin] RBAC: Adding query filter params (name=network) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:30.972605 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-616e96a5-feae-4cf8-9645-e4340b0a6654 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:18:30.972605 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:30.972605 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:30.972605 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:30.972605 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:30.972605 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:30.972605 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:30.972605 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:30.972605 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:30.972605 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:30.972605 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:30.972605 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:30.972605 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:18:30.972605 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:18:30.972605 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:18:30.972605 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:18:30.974632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:18:30.974632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:18:30.974632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:18:30.974632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:18:30.974632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:30.974632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:30.974632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:30.974632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:30.974632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:30.974632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:30.974632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:30.974632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:30.974632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:30.974632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:30.974632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:30.974632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:30.976591 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:30.976591 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:30.976591 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:30.976591 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:30.976591 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:18:30.976591 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:18:30.976591 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:18:30.976591 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:18:30.976591 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:18:30.976591 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:18:30.976591 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:18:30.976591 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:30.976591 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 400/810] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 21:18:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:30.978089 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e96a5-feae-4cf8-9645-e4340b0a6654 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:30.978342 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ca9789-db11-45eb-bc7c-f729555bd821 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:30.981510 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ca9789-db11-45eb-bc7c-f729555bd821 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:30.988612 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 411/811] 10.222.0.65 () {56 vars in 1112 bytes} [Thu May 19 21:18:30 2022] GET /identity/v3/services?name=network => generated 124 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:30.992026 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8b893-e27c-41dd-b7c9-8ee7987aaae0 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:30.992851 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8b893-e27c-41dd-b7c9-8ee7987aaae0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:30.993767 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8b893-e27c-41dd-b7c9-8ee7987aaae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:31.018229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-edf8b893-e27c-41dd-b7c9-8ee7987aaae0 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:31.020454 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-edf8b893-e27c-41dd-b7c9-8ee7987aaae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:31.136338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 412/812] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:18:31.136338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8b893-e27c-41dd-b7c9-8ee7987aaae0 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:31.157383 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e96a5-feae-4cf8-9645-e4340b0a6654 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:31.183143 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:31.221686 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b9946-b3ad-414e-bf25-ab0c55e37614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:31.222892 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b9946-b3ad-414e-bf25-ab0c55e37614 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:31.223374 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b9946-b3ad-414e-bf25-ab0c55e37614 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:31.223858 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b9946-b3ad-414e-bf25-ab0c55e37614 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:31.234213 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:31.237471 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b9946-b3ad-414e-bf25-ab0c55e37614 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:31.240242 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b9946-b3ad-414e-bf25-ab0c55e37614 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:31.268052 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-d56b9946-b3ad-414e-bf25-ab0c55e37614 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:31.268052 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:31.268052 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:31.268052 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:31.268052 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:31.268052 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:31.268052 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:31.268052 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:31.268052 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:31.268052 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:31.268052 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:31.268052 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:31.268052 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:18:31.268052 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:18:31.268052 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:18:31.269192 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:18:31.269192 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:31.269192 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:31.269192 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:31.269192 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:31.269192 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:31.269192 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:31.269192 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:31.269192 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:31.269192 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:31.269192 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:31.269192 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:31.269192 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:31.269192 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:31.269192 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:31.269192 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:31.269192 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:31.270086 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:18:31.270086 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:18:31.270086 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:18:31.270086 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:18:31.270086 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:18:31.270086 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:18:31.270086 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:31.270086 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:31.270864 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 401/813] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:18:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:31.272401 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b9946-b3ad-414e-bf25-ab0c55e37614 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:31.291312 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8b893-e27c-41dd-b7c9-8ee7987aaae0 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:31.341762 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf47cf8-ec57-4cc0-bd81-5a8bec30813f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:31.343006 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf47cf8-ec57-4cc0-bd81-5a8bec30813f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:31.343469 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf47cf8-ec57-4cc0-bd81-5a8bec30813f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:31.344600 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf47cf8-ec57-4cc0-bd81-5a8bec30813f None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:31.346192 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf47cf8-ec57-4cc0-bd81-5a8bec30813f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:31.346914 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf47cf8-ec57-4cc0-bd81-5a8bec30813f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:31.350368 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf47cf8-ec57-4cc0-bd81-5a8bec30813f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:31.374321 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 413/814] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:18:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:31.374321 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf47cf8-ec57-4cc0-bd81-5a8bec30813f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:31.447720 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b9946-b3ad-414e-bf25-ab0c55e37614 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:31.504065 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea55f9cf-fa32-4ceb-b548-8b1cf2cde31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:31.504992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55f9cf-fa32-4ceb-b548-8b1cf2cde31f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:31.505450 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55f9cf-fa32-4ceb-b548-8b1cf2cde31f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:31.505937 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55f9cf-fa32-4ceb-b548-8b1cf2cde31f None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:31.507216 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea55f9cf-fa32-4ceb-b548-8b1cf2cde31f None admin] RBAC: Adding query filter params (type=network) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:31.507715 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea55f9cf-fa32-4ceb-b548-8b1cf2cde31f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:31.509708 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea55f9cf-fa32-4ceb-b548-8b1cf2cde31f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:31.524931 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 402/815] 10.222.0.65 () {56 vars in 1112 bytes} [Thu May 19 21:18:31 2022] GET /identity/v3/services?type=network => generated 350 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:31.524931 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea55f9cf-fa32-4ceb-b548-8b1cf2cde31f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:31.557303 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf47cf8-ec57-4cc0-bd81-5a8bec30813f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:31.584413 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:31.603692 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:31.623050 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c852e12a-c3c5-4671-bcef-f8cd4792ab11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:31.624327 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852e12a-c3c5-4671-bcef-f8cd4792ab11 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:31.624785 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852e12a-c3c5-4671-bcef-f8cd4792ab11 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:31.625235 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852e12a-c3c5-4671-bcef-f8cd4792ab11 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:31.626375 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c852e12a-c3c5-4671-bcef-f8cd4792ab11 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:31.628629 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c852e12a-c3c5-4671-bcef-f8cd4792ab11 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:31.649403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-c852e12a-c3c5-4671-bcef-f8cd4792ab11 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 21:18:31.649403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:31.649403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:18:31.649403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:18:31.649403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:18:31.649403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:18:31.649403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:18:31.649403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:18:31.649403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 21:18:31.649403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:31.649403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:18:31.649403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:31.649403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:31.649403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:31.649403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:31.649403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:31.651542 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:31.651542 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:31.651542 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:31.651542 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:31.651542 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:31.651542 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:31.651542 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:31.651542 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:18:31.651542 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:18:31.651542 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:18:31.651542 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:18:31.651542 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:31.651542 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:31.651542 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:31.651542 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:31.651542 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:31.653177 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:31.653177 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:31.653177 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:31.653177 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:31.653177 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:31.653177 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:31.653177 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:31.653177 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:31.653177 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:31.653177 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:31.653177 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:31.653177 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:18:31.653177 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:18:31.653177 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 21:18:31.653177 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:31.659471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 414/816] 10.222.0.65 () {56 vars in 1129 bytes} [Thu May 19 21:18:31 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:31.661179 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c852e12a-c3c5-4671-bcef-f8cd4792ab11 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:31.705826 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea55f9cf-fa32-4ceb-b548-8b1cf2cde31f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:31.766200 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b0f28-6370-4c89-af55-b7e05c094f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:31.767060 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b0f28-6370-4c89-af55-b7e05c094f4f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:31.767442 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b0f28-6370-4c89-af55-b7e05c094f4f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:31.767896 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b0f28-6370-4c89-af55-b7e05c094f4f None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:31.769261 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826b0f28-6370-4c89-af55-b7e05c094f4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:31.769656 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826b0f28-6370-4c89-af55-b7e05c094f4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:31.772325 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826b0f28-6370-4c89-af55-b7e05c094f4f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:31.803323 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 403/817] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 21:18:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:31.803323 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b0f28-6370-4c89-af55-b7e05c094f4f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:31.840420 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c852e12a-c3c5-4671-bcef-f8cd4792ab11 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:31.907460 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ece105-3580-49d0-bb4e-a7573bdd628c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:31.908738 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ece105-3580-49d0-bb4e-a7573bdd628c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:31.909246 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ece105-3580-49d0-bb4e-a7573bdd628c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:31.909670 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ece105-3580-49d0-bb4e-a7573bdd628c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:31.932404 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ece105-3580-49d0-bb4e-a7573bdd628c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:31.935265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ece105-3580-49d0-bb4e-a7573bdd628c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:31.949171 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:31.959059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b0f28-6370-4c89-af55-b7e05c094f4f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:31.969636 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-12ece105-3580-49d0-bb4e-a7573bdd628c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:31.969636 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:31.969636 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:31.969636 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:31.969636 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:31.969636 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:31.969636 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:31.969636 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:31.969636 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:31.969636 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:31.969636 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:31.969636 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:31.969636 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:31.969636 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:31.969636 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:31.971806 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:31.971806 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:31.971806 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:31.971806 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:31.971806 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:31.971806 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:31.971806 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:31.971806 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:31.971806 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:31.971806 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:31.971806 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:31.971806 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:31.971806 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:31.971806 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:31.971806 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:31.971806 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:31.971806 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:31.974099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:31.974099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:31.974099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:31.974099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:31.974099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 415/818] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:31.974099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ece105-3580-49d0-bb4e-a7573bdd628c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:32.028743 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbd12d-f54f-4b2f-91bc-123d5d77901c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:32.028743 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbd12d-f54f-4b2f-91bc-123d5d77901c None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:32.029204 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbd12d-f54f-4b2f-91bc-123d5d77901c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:32.029602 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbd12d-f54f-4b2f-91bc-123d5d77901c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:32.030672 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dbd12d-f54f-4b2f-91bc-123d5d77901c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:32.032300 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dbd12d-f54f-4b2f-91bc-123d5d77901c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:32.044016 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:32.074283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 404/819] 10.222.0.65 () {60 vars in 1144 bytes} [Thu May 19 21:18:31 2022] POST /identity/v3/endpoints => generated 327 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:18:32.074283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbd12d-f54f-4b2f-91bc-123d5d77901c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:32.144578 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ece105-3580-49d0-bb4e-a7573bdd628c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:32.219477 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86765547-1ac9-4ffe-ae44-f30903d65fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:32.221227 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86765547-1ac9-4ffe-ae44-f30903d65fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:32.222204 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86765547-1ac9-4ffe-ae44-f30903d65fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:32.224283 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86765547-1ac9-4ffe-ae44-f30903d65fa6 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:32.226024 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86765547-1ac9-4ffe-ae44-f30903d65fa6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:32.226801 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86765547-1ac9-4ffe-ae44-f30903d65fa6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:32.230547 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86765547-1ac9-4ffe-ae44-f30903d65fa6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:32.240896 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 416/820] 10.222.0.65 () {56 vars in 1124 bytes} [Thu May 19 21:18:31 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:32.241878 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86765547-1ac9-4ffe-ae44-f30903d65fa6 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:32.291136 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbd12d-f54f-4b2f-91bc-123d5d77901c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:32.358144 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f7e43-971e-4e1f-bad3-ceec2372683c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:32.359781 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f7e43-971e-4e1f-bad3-ceec2372683c None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:32.360272 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f7e43-971e-4e1f-bad3-ceec2372683c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:32.360799 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f7e43-971e-4e1f-bad3-ceec2372683c None admin] PATH_INFO: `/v3/projects/c4940befa9c4405699c72deedc8a2127/users/7e07984b36934aff9052541ef2a56f36/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:32.366102 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0f7e43-971e-4e1f-bad3-ceec2372683c None admin] RBAC: Authorizing `identity:create_grant(project_id=c4940befa9c4405699c72deedc8a2127, user_id=7e07984b36934aff9052541ef2a56f36, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:32.369153 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0f7e43-971e-4e1f-bad3-ceec2372683c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:32.381291 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:32.387480 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86765547-1ac9-4ffe-ae44-f30903d65fa6 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:32.397837 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 405/821] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 21:18:32 2022] PUT /identity/v3/projects/c4940befa9c4405699c72deedc8a2127/users/7e07984b36934aff9052541ef2a56f36/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:18:32.398930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f7e43-971e-4e1f-bad3-ceec2372683c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:32.419542 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:32.476225 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfada65-bcc3-4ab1-9283-fc1d34f8257d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:32.478413 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfada65-bcc3-4ab1-9283-fc1d34f8257d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:32.479409 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfada65-bcc3-4ab1-9283-fc1d34f8257d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:32.480755 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfada65-bcc3-4ab1-9283-fc1d34f8257d None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:32.483070 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfada65-bcc3-4ab1-9283-fc1d34f8257d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:32.484471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfada65-bcc3-4ab1-9283-fc1d34f8257d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:32.488746 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfada65-bcc3-4ab1-9283-fc1d34f8257d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:32.512762 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 417/822] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:32.513892 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfada65-bcc3-4ab1-9283-fc1d34f8257d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:32.592058 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f7e43-971e-4e1f-bad3-ceec2372683c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:32.659719 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfada65-bcc3-4ab1-9283-fc1d34f8257d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:32.660060 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc290e4d-fd84-4348-a2ee-d13dd9b2fdf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:32.660965 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc290e4d-fd84-4348-a2ee-d13dd9b2fdf5 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:32.661425 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc290e4d-fd84-4348-a2ee-d13dd9b2fdf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:32.661985 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc290e4d-fd84-4348-a2ee-d13dd9b2fdf5 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:32.663031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc290e4d-fd84-4348-a2ee-d13dd9b2fdf5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:32.664055 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc290e4d-fd84-4348-a2ee-d13dd9b2fdf5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:32.666949 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc290e4d-fd84-4348-a2ee-d13dd9b2fdf5 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:32.678898 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 406/823] 10.222.0.65 () {56 vars in 1124 bytes} [Thu May 19 21:18:32 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:32.680388 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc290e4d-fd84-4348-a2ee-d13dd9b2fdf5 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:32.740617 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d30ce3a-e0b0-47ef-bd36-30b25af48bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:32.741730 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d30ce3a-e0b0-47ef-bd36-30b25af48bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:32.742218 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d30ce3a-e0b0-47ef-bd36-30b25af48bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:32.742778 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d30ce3a-e0b0-47ef-bd36-30b25af48bb3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:32.754801 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d30ce3a-e0b0-47ef-bd36-30b25af48bb3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:32.756866 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d30ce3a-e0b0-47ef-bd36-30b25af48bb3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:32.791843 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-8d30ce3a-e0b0-47ef-bd36-30b25af48bb3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 21:18:32.791843 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:32.791843 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:32.791843 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:32.791843 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:32.791843 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:32.791843 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:32.791843 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:32.791843 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:32.791843 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:32.791843 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:32.791843 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:32.791843 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:18:32.791843 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:18:32.791843 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:18:32.791843 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:18:32.793538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:32.793538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:32.793538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:18:32.793538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:32.793538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:18:32.793538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:32.793538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:32.793538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:32.793538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:32.793538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:32.793538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:32.793538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:32.793538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:32.793538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:32.793538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:32.793538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:32.793538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:32.793538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:32.795355 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:32.795355 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:32.795355 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:18:32.795355 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:18:32.795355 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:18:32.795355 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:18:32.795355 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:18:32.795355 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:18:32.795355 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 21:18:32.795355 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:32.799995 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 418/824] 10.222.0.65 () {56 vars in 1093 bytes} [Thu May 19 21:18:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:32.811464 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d30ce3a-e0b0-47ef-bd36-30b25af48bb3 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:32.841932 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:32.868637 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:32.894559 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc290e4d-fd84-4348-a2ee-d13dd9b2fdf5 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:32.960517 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c02f9c-eef7-4383-a39c-96315a4fb577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:32.961437 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c02f9c-eef7-4383-a39c-96315a4fb577 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:32.961553 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c02f9c-eef7-4383-a39c-96315a4fb577 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:32.961985 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c02f9c-eef7-4383-a39c-96315a4fb577 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:32.963466 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c02f9c-eef7-4383-a39c-96315a4fb577 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:32.966104 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c02f9c-eef7-4383-a39c-96315a4fb577 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:33.010411 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 407/825] 10.222.0.65 () {60 vars in 1144 bytes} [Thu May 19 21:18:32 2022] POST /identity/v3/endpoints => generated 346 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:18:33.028541 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d30ce3a-e0b0-47ef-bd36-30b25af48bb3 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:33.119809 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1492c31-6b9e-4d16-84c1-443baaec8dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:33.121520 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1492c31-6b9e-4d16-84c1-443baaec8dc0 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:33.122459 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1492c31-6b9e-4d16-84c1-443baaec8dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:33.123257 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1492c31-6b9e-4d16-84c1-443baaec8dc0 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:33.125325 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1492c31-6b9e-4d16-84c1-443baaec8dc0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:33.126223 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1492c31-6b9e-4d16-84c1-443baaec8dc0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:33.129223 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1492c31-6b9e-4d16-84c1-443baaec8dc0 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:33.163978 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 419/826] 10.222.0.65 () {56 vars in 1133 bytes} [Thu May 19 21:18:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:33.172204 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c02f9c-eef7-4383-a39c-96315a4fb577 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:33.345125 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b302c6-4fc7-4c95-9033-715786eb5113 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:33.345433 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b302c6-4fc7-4c95-9033-715786eb5113 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:33.345766 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b302c6-4fc7-4c95-9033-715786eb5113 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:33.347424 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 420/827] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:18:33.353895 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da52340-3a2c-4696-950d-56826d3ade2f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:33.354329 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da52340-3a2c-4696-950d-56826d3ade2f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:33.354510 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da52340-3a2c-4696-950d-56826d3ade2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:33.369410 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c02f9c-eef7-4383-a39c-96315a4fb577 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:33.392336 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-6da52340-3a2c-4696-950d-56826d3ade2f None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:33.393502 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6da52340-3a2c-4696-950d-56826d3ade2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:33.412998 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:33.420871 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e328ac61-cc48-4249-a6d0-7a2722872341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:33.423028 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328ac61-cc48-4249-a6d0-7a2722872341 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:33.423930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328ac61-cc48-4249-a6d0-7a2722872341 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:33.424473 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328ac61-cc48-4249-a6d0-7a2722872341 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:33.498431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 421/828] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:33 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:18:33.540268 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da52340-3a2c-4696-950d-56826d3ade2f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:33.545154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:33.563490 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e328ac61-cc48-4249-a6d0-7a2722872341 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:33.565340 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e328ac61-cc48-4249-a6d0-7a2722872341 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:33.683999 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 408/829] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:18:33 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 21:18:33.684947 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e328ac61-cc48-4249-a6d0-7a2722872341 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:33.719555 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:33.745712 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da52340-3a2c-4696-950d-56826d3ade2f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:33.803660 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e328ac61-cc48-4249-a6d0-7a2722872341 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:33.816928 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710c5e8-3970-4a6f-8546-379efdebb37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:33.817761 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710c5e8-3970-4a6f-8546-379efdebb37a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:33.818121 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710c5e8-3970-4a6f-8546-379efdebb37a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:33.819073 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710c5e8-3970-4a6f-8546-379efdebb37a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:33.835561 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c710c5e8-3970-4a6f-8546-379efdebb37a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:18:33.851435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-c710c5e8-3970-4a6f-8546-379efdebb37a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:18:33.851435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:33.851435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:33.851435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:33.851435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:33.851435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:33.851435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:33.851435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:33.851435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:33.851435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:33.851435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:33.851435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:33.851435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:18:33.851435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:18:33.851435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:18:33.851435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:18:33.852445 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:18:33.852445 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:18:33.852445 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:18:33.852445 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:18:33.852445 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:33.852445 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:33.852445 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:33.852445 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:33.852445 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:33.852445 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:33.852445 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:33.852445 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:33.852445 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:33.852445 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:33.852445 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:33.852445 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:33.853314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:33.853314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:33.853314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:33.853314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:33.853314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:18:33.853314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:18:33.853314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:18:33.853314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:18:33.853314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:18:33.853314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:18:33.853314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:18:33.853314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:33.862359 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 422/830] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 21:18:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:18:33.866495 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710c5e8-3970-4a6f-8546-379efdebb37a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:33.884386 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd8c78-e4db-4ab6-a9c1-d281f4792212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:33.886673 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd8c78-e4db-4ab6-a9c1-d281f4792212 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:33.887260 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd8c78-e4db-4ab6-a9c1-d281f4792212 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:33.887725 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd8c78-e4db-4ab6-a9c1-d281f4792212 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:33.915664 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cd8c78-e4db-4ab6-a9c1-d281f4792212 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:33.920496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cd8c78-e4db-4ab6-a9c1-d281f4792212 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:33.957556 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:33.970840 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-39cd8c78-e4db-4ab6-a9c1-d281f4792212 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:33.970840 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:33.970840 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:33.970840 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:33.970840 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:33.970840 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:33.970840 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:33.970840 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:33.970840 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:33.970840 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:33.970840 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:33.970840 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:33.970840 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:33.970840 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:33.970840 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:33.972276 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:33.972276 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:33.972276 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:33.972276 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:33.972276 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:33.972276 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:33.972276 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:33.972276 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:33.972276 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:33.972276 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:33.972276 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:33.972276 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:33.972276 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:33.972276 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:33.972276 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:33.972276 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:33.972276 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:33.973296 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:33.973296 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:33.973296 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:33.973296 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:33.977472 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 409/831] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:33.982910 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd8c78-e4db-4ab6-a9c1-d281f4792212 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:34.022870 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710c5e8-3970-4a6f-8546-379efdebb37a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:34.064512 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1779dd-f40e-403a-b1e2-0e36611e2b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:34.065523 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1779dd-f40e-403a-b1e2-0e36611e2b8c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:34.065809 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1779dd-f40e-403a-b1e2-0e36611e2b8c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:34.066085 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1779dd-f40e-403a-b1e2-0e36611e2b8c None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:34.067171 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1779dd-f40e-403a-b1e2-0e36611e2b8c None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:34.067171 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1779dd-f40e-403a-b1e2-0e36611e2b8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:34.068748 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1779dd-f40e-403a-b1e2-0e36611e2b8c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:34.078569 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 423/832] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 21:18:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:18:34.087707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1779dd-f40e-403a-b1e2-0e36611e2b8c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:34.126350 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd8c78-e4db-4ab6-a9c1-d281f4792212 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:34.141009 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:34.187763 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38b84b8-108e-4b9c-a4a9-3fd63cf0f76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:34.188843 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38b84b8-108e-4b9c-a4a9-3fd63cf0f76b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:34.189317 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38b84b8-108e-4b9c-a4a9-3fd63cf0f76b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:34.189794 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38b84b8-108e-4b9c-a4a9-3fd63cf0f76b None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:34.191283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38b84b8-108e-4b9c-a4a9-3fd63cf0f76b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:34.191827 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38b84b8-108e-4b9c-a4a9-3fd63cf0f76b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:34.194946 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38b84b8-108e-4b9c-a4a9-3fd63cf0f76b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:34.216922 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 410/833] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:18:34.223118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38b84b8-108e-4b9c-a4a9-3fd63cf0f76b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:34.225810 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1779dd-f40e-403a-b1e2-0e36611e2b8c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:34.268826 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:34.279297 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41a2d2-11f6-4be8-9751-c92c061b4c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:34.280567 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41a2d2-11f6-4be8-9751-c92c061b4c26 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:34.281029 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41a2d2-11f6-4be8-9751-c92c061b4c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:34.281470 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41a2d2-11f6-4be8-9751-c92c061b4c26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:34.294613 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec41a2d2-11f6-4be8-9751-c92c061b4c26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:34.296476 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec41a2d2-11f6-4be8-9751-c92c061b4c26 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:34.316092 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-ec41a2d2-11f6-4be8-9751-c92c061b4c26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:34.316092 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:34.316092 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:34.316092 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:34.316092 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:34.316092 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:34.316092 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:34.316092 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:34.316092 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:34.316092 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:34.316092 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:34.316092 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:34.316092 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:34.316092 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:34.316092 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:34.317055 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:34.317055 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:34.317055 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:34.317055 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:34.317055 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:34.317055 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:34.317055 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:34.317055 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:34.317055 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:34.317055 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:34.317055 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:34.317055 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:34.317055 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:34.317055 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:34.317055 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:34.317055 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:34.317055 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:34.317930 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:34.317930 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:34.317930 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:34.317930 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:34.320930 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 424/834] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:34.329988 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41a2d2-11f6-4be8-9751-c92c061b4c26 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:34.384244 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38b84b8-108e-4b9c-a4a9-3fd63cf0f76b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:34.455993 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b17e0f-a7f3-4790-9da9-307477ed94a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:34.457497 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b17e0f-a7f3-4790-9da9-307477ed94a0 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:34.457956 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b17e0f-a7f3-4790-9da9-307477ed94a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:34.458557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b17e0f-a7f3-4790-9da9-307477ed94a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:34.472993 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b17e0f-a7f3-4790-9da9-307477ed94a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:34.475941 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b17e0f-a7f3-4790-9da9-307477ed94a0 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:34.491326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41a2d2-11f6-4be8-9751-c92c061b4c26 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:34.510283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-d6b17e0f-a7f3-4790-9da9-307477ed94a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:34.510283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:34.510283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:34.510283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:34.510283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:34.510283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:34.510283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:34.510283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:34.510283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:34.510283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:34.510283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:34.510283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:34.510283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:18:34.510283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:18:34.510283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:18:34.511241 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:18:34.511241 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:34.511241 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:34.511241 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:34.511241 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:34.511241 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:34.511241 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:34.511241 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:34.511241 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:34.511241 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:34.511241 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:34.511241 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:34.511241 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:34.511241 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:34.511241 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:34.511241 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:34.511241 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:34.512134 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:18:34.512134 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:18:34.512134 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:18:34.512134 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:18:34.512134 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:18:34.512134 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:18:34.512134 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:34.512134 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:34.512134 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 411/835] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:18:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:34.519300 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b17e0f-a7f3-4790-9da9-307477ed94a0 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:34.524066 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:34.534065 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b429aa3-ab0c-4549-8bc8-3a269163ed38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:34.534790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b429aa3-ab0c-4549-8bc8-3a269163ed38 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:34.535071 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b429aa3-ab0c-4549-8bc8-3a269163ed38 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:34.535667 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b429aa3-ab0c-4549-8bc8-3a269163ed38 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:34.536095 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b429aa3-ab0c-4549-8bc8-3a269163ed38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:34.536403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b429aa3-ab0c-4549-8bc8-3a269163ed38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:34.538797 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b429aa3-ab0c-4549-8bc8-3a269163ed38 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:34.552660 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 425/836] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:34.561851 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b429aa3-ab0c-4549-8bc8-3a269163ed38 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:34.640853 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:34.649930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b17e0f-a7f3-4790-9da9-307477ed94a0 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:34.690341 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb08201-796a-4bfc-806d-735b10293203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:34.691467 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb08201-796a-4bfc-806d-735b10293203 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:34.691977 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb08201-796a-4bfc-806d-735b10293203 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:34.692440 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb08201-796a-4bfc-806d-735b10293203 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:34.693545 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb08201-796a-4bfc-806d-735b10293203 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:34.694053 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb08201-796a-4bfc-806d-735b10293203 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:34.695853 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb08201-796a-4bfc-806d-735b10293203 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:34.714780 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 412/837] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 21:18:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:34.716035 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb08201-796a-4bfc-806d-735b10293203 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:34.717576 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b429aa3-ab0c-4549-8bc8-3a269163ed38 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:34.758918 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2317f654-dcf5-4996-8469-ee86e89367f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:34.759676 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317f654-dcf5-4996-8469-ee86e89367f8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:34.759963 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317f654-dcf5-4996-8469-ee86e89367f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:34.760237 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317f654-dcf5-4996-8469-ee86e89367f8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:34.768676 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2317f654-dcf5-4996-8469-ee86e89367f8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:34.770143 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2317f654-dcf5-4996-8469-ee86e89367f8 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:34.788775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-2317f654-dcf5-4996-8469-ee86e89367f8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 21:18:34.788775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:34.788775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:34.788775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:34.788775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:34.788775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:34.788775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:34.788775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:34.788775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:34.788775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:34.788775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:34.788775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:34.788775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:18:34.788775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:18:34.788775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:18:34.788775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:18:34.789811 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:34.789811 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:34.789811 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:18:34.789811 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:34.789811 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:18:34.789811 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:34.789811 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:34.789811 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:34.789811 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:34.789811 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:34.789811 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:34.789811 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:34.789811 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:34.789811 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:34.789811 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:34.789811 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:34.789811 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:34.789811 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:34.790793 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:34.790793 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:34.790793 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:18:34.790793 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:18:34.790793 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:18:34.790793 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:18:34.790793 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:18:34.790793 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:18:34.790793 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 21:18:34.790793 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:34.790793 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 426/838] 10.222.0.65 () {56 vars in 1117 bytes} [Thu May 19 21:18:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:34.793025 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:34.802276 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2317f654-dcf5-4996-8469-ee86e89367f8 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:34.863223 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb08201-796a-4bfc-806d-735b10293203 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:34.904725 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c50ebf-1a22-4711-b694-637f5401b09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:34.905763 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c50ebf-1a22-4711-b694-637f5401b09d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:34.906268 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c50ebf-1a22-4711-b694-637f5401b09d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:34.906877 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c50ebf-1a22-4711-b694-637f5401b09d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:34.908039 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c50ebf-1a22-4711-b694-637f5401b09d None admin] RBAC: Adding query filter params (role.id=2b714f2e5c6c469f8fa037bda1ed16f6, scope.project.id=c4940befa9c4405699c72deedc8a2127, user.id=e254671b21fe4661a381b9521810cf80) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:34.908558 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c50ebf-1a22-4711-b694-637f5401b09d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:34.910221 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c50ebf-1a22-4711-b694-637f5401b09d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:34.918843 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 413/839] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:18:34 2022] GET /identity/v3/role_assignments?user.id=e254671b21fe4661a381b9521810cf80&scope.project.id=c4940befa9c4405699c72deedc8a2127&role.id=2b714f2e5c6c469f8fa037bda1ed16f6 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:35.002736 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:35.010296 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2317f654-dcf5-4996-8469-ee86e89367f8 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:35.079817 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9036fa76-ad99-4a00-a724-34b62bd951ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:35.081020 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036fa76-ad99-4a00-a724-34b62bd951ff None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:35.081520 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036fa76-ad99-4a00-a724-34b62bd951ff None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:35.082015 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036fa76-ad99-4a00-a724-34b62bd951ff None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:35.083710 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9036fa76-ad99-4a00-a724-34b62bd951ff None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:35.084597 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9036fa76-ad99-4a00-a724-34b62bd951ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:35.087086 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9036fa76-ad99-4a00-a724-34b62bd951ff None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:35.130206 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 427/840] 10.222.0.65 () {56 vars in 1149 bytes} [Thu May 19 21:18:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:35.135123 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c50ebf-1a22-4711-b694-637f5401b09d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:35.331659 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c50ebf-1a22-4711-b694-637f5401b09d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:35.387227 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:35.401641 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dea22e-c549-45cf-b948-81c293a9b56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:35.402796 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dea22e-c549-45cf-b948-81c293a9b56b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:35.403291 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dea22e-c549-45cf-b948-81c293a9b56b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:35.403782 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dea22e-c549-45cf-b948-81c293a9b56b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:35.500466 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dea22e-c549-45cf-b948-81c293a9b56b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:35.501955 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dea22e-c549-45cf-b948-81c293a9b56b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:35.551483 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6519e86-a7b8-41a0-86b3-dead0c8378f9 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:35.551839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6519e86-a7b8-41a0-86b3-dead0c8378f9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:35.552054 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6519e86-a7b8-41a0-86b3-dead0c8378f9 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:35.554078 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 428/841] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:18:35.560265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbe1be-4131-4a00-bdee-1877982dfe6e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:35.560585 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbe1be-4131-4a00-bdee-1877982dfe6e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:35.560822 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbe1be-4131-4a00-bdee-1877982dfe6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:35.584791 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b2bbe1be-4131-4a00-bdee-1877982dfe6e None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:35.587424 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b2bbe1be-4131-4a00-bdee-1877982dfe6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:35.665105 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 414/842] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:18:35 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 21:18:35.671327 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dea22e-c549-45cf-b948-81c293a9b56b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:35.679954 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 429/843] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:35 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:18:35.702442 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:35.703224 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bbe1be-4131-4a00-bdee-1877982dfe6e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:35.790454 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dea22e-c549-45cf-b948-81c293a9b56b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:35.798821 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:35.816888 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bbe1be-4131-4a00-bdee-1877982dfe6e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:35.826387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0bbf5-a3d5-452d-aed0-76a7ccf577c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:35.827204 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0bbf5-a3d5-452d-aed0-76a7ccf577c8 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:35.827487 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0bbf5-a3d5-452d-aed0-76a7ccf577c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:35.827763 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0bbf5-a3d5-452d-aed0-76a7ccf577c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:35.839286 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d0bbf5-a3d5-452d-aed0-76a7ccf577c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:35.840712 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d0bbf5-a3d5-452d-aed0-76a7ccf577c8 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:35.863513 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa8ca05-82f1-4816-9caf-ba9bbf5aec87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:35.864681 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa8ca05-82f1-4816-9caf-ba9bbf5aec87 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:35.865555 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa8ca05-82f1-4816-9caf-ba9bbf5aec87 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:35.865999 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa8ca05-82f1-4816-9caf-ba9bbf5aec87 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:35.868083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-51d0bbf5-a3d5-452d-aed0-76a7ccf577c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:35.868083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:35.868083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:35.868083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:35.868083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:35.868083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:35.868083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:35.868083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:35.868083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:35.868083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:35.868083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:35.868083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:35.868083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:35.868083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:35.868083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:35.868985 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:35.868985 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:35.868985 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:35.868985 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:35.868985 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:35.868985 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:35.868985 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:35.868985 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:35.868985 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:35.868985 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:35.868985 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:35.868985 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:35.868985 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:35.868985 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:35.868985 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:35.868985 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:35.868985 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:35.869875 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:35.869875 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:35.869875 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:35.869875 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:35.869875 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 415/844] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:35.870701 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c1d3e-4f62-4196-9644-a6a3e93bed6b None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:35.870989 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c1d3e-4f62-4196-9644-a6a3e93bed6b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:35.871269 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c1d3e-4f62-4196-9644-a6a3e93bed6b None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:35.872762 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 416/845] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:18:35.877791 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c1d3e-4f62-4196-9644-a6a3e93bed6b None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:35.886882 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa8ca05-82f1-4816-9caf-ba9bbf5aec87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:18:35.895705 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-7aa8ca05-82f1-4816-9caf-ba9bbf5aec87 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:18:35.895705 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:35.895705 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:35.895705 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:35.895705 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:35.895705 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:35.895705 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:35.895705 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:35.895705 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:35.895705 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:35.895705 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:35.895705 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:35.895705 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:18:35.895705 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:18:35.895705 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:18:35.895705 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:18:35.897255 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:18:35.897255 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:18:35.897255 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:18:35.897255 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:18:35.897255 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:35.897255 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:35.897255 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:35.897255 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:35.897255 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:35.897255 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:35.897255 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:35.897255 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:35.897255 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:35.897255 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:35.897255 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:35.897255 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:35.898270 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:35.898270 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:35.898270 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:35.898270 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:35.898270 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:18:35.898270 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:18:35.898270 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:18:35.898270 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:18:35.898270 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:18:35.898270 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:18:35.898270 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:18:35.898270 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:35.900695 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 430/846] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 21:18:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:18:35.902186 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b187d5b3-2a72-45fd-8587-b3e5b3003d33 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:35.902512 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b187d5b3-2a72-45fd-8587-b3e5b3003d33 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:35.902788 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b187d5b3-2a72-45fd-8587-b3e5b3003d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:35.921178 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b187d5b3-2a72-45fd-8587-b3e5b3003d33 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:35.922416 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b187d5b3-2a72-45fd-8587-b3e5b3003d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:35.977568 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:35.996478 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 431/847] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:35 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:18:35.997219 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b187d5b3-2a72-45fd-8587-b3e5b3003d33 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:36.013879 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c1d3e-4f62-4196-9644-a6a3e93bed6b None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:36.068938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a219a9-d4cc-4e05-aee6-42328cd795b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:36.069637 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a219a9-d4cc-4e05-aee6-42328cd795b9 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:36.069921 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a219a9-d4cc-4e05-aee6-42328cd795b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:36.070200 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a219a9-d4cc-4e05-aee6-42328cd795b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:36.071111 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a219a9-d4cc-4e05-aee6-42328cd795b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:36.071423 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a219a9-d4cc-4e05-aee6-42328cd795b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:36.072820 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a219a9-d4cc-4e05-aee6-42328cd795b9 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:36.086025 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 417/848] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:36.086790 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a219a9-d4cc-4e05-aee6-42328cd795b9 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:36.107410 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:36.145959 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b187d5b3-2a72-45fd-8587-b3e5b3003d33 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:36.201729 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be8744-7887-4e77-b6b3-a9817c5b4a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:36.202824 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be8744-7887-4e77-b6b3-a9817c5b4a73 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:36.203543 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be8744-7887-4e77-b6b3-a9817c5b4a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:36.203543 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be8744-7887-4e77-b6b3-a9817c5b4a73 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:36.204590 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47be8744-7887-4e77-b6b3-a9817c5b4a73 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:36.205090 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47be8744-7887-4e77-b6b3-a9817c5b4a73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:36.207299 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47be8744-7887-4e77-b6b3-a9817c5b4a73 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:36.221312 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 432/849] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:18:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:36.222786 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be8744-7887-4e77-b6b3-a9817c5b4a73 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:36.260560 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a219a9-d4cc-4e05-aee6-42328cd795b9 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:36.328321 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf5ab16-74a5-438b-9893-68fa787faecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:36.329735 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5ab16-74a5-438b-9893-68fa787faecb None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:36.330227 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5ab16-74a5-438b-9893-68fa787faecb None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:36.330755 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5ab16-74a5-438b-9893-68fa787faecb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:36.331959 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf5ab16-74a5-438b-9893-68fa787faecb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:36.335564 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf5ab16-74a5-438b-9893-68fa787faecb None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:36.353495 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be8744-7887-4e77-b6b3-a9817c5b4a73 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:36.359413 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-daf5ab16-74a5-438b-9893-68fa787faecb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 21:18:36.359413 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:36.359413 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:18:36.359413 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:18:36.359413 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:18:36.359413 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:18:36.359413 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:18:36.359413 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:18:36.359413 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 21:18:36.359413 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:36.359413 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:18:36.359413 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:36.359413 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:36.359413 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:36.359413 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:36.359413 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:36.359413 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:36.361374 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:36.361374 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:36.361374 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:36.361374 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:36.361374 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:36.361374 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:36.361374 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:18:36.361374 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:18:36.361374 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:18:36.361374 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:18:36.361374 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:36.361374 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:36.361374 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:36.361374 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:36.361374 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:36.361374 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:36.363824 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:36.363824 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:36.363824 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:36.363824 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:36.363824 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:36.363824 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:36.363824 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:36.363824 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:36.363824 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:36.363824 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:36.363824 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:18:36.363824 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:18:36.363824 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 21:18:36.363824 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:36.370459 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 418/850] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 21:18:36 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:36.371841 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf5ab16-74a5-438b-9893-68fa787faecb None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:36.378288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:36.407809 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c9e7e-b0e2-4dd0-b91c-fa58545329b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:36.409258 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c9e7e-b0e2-4dd0-b91c-fa58545329b1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:36.409682 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c9e7e-b0e2-4dd0-b91c-fa58545329b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:36.410092 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c9e7e-b0e2-4dd0-b91c-fa58545329b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:36.419695 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971c9e7e-b0e2-4dd0-b91c-fa58545329b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:36.421774 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971c9e7e-b0e2-4dd0-b91c-fa58545329b1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:36.424359 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:36.445371 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-971c9e7e-b0e2-4dd0-b91c-fa58545329b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:36.445371 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:36.445371 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:36.445371 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:36.445371 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:36.445371 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:36.445371 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:36.445371 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:36.445371 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:36.445371 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:36.445371 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:36.445371 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:36.445371 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:18:36.445371 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:18:36.445371 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:18:36.447195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:18:36.447195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:36.447195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:36.447195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:36.447195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:36.447195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:36.447195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:36.447195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:36.447195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:36.447195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:36.447195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:36.447195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:36.447195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:36.447195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:36.447195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:36.447195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:36.447195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:36.448934 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:18:36.448934 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:18:36.448934 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:18:36.448934 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:18:36.448934 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:18:36.448934 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:18:36.448934 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:36.448934 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:36.448934 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 433/851] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:18:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:36.449878 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c9e7e-b0e2-4dd0-b91c-fa58545329b1 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:36.590752 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf5ab16-74a5-438b-9893-68fa787faecb None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:36.608299 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c9e7e-b0e2-4dd0-b91c-fa58545329b1 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:36.653960 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8a5c8-b3b4-429d-9410-2ef2d4d5cc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:36.654630 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8a5c8-b3b4-429d-9410-2ef2d4d5cc67 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:36.654923 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8a5c8-b3b4-429d-9410-2ef2d4d5cc67 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:36.655212 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8a5c8-b3b4-429d-9410-2ef2d4d5cc67 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:36.655950 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc8a5c8-b3b4-429d-9410-2ef2d4d5cc67 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:36.656237 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc8a5c8-b3b4-429d-9410-2ef2d4d5cc67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:36.657668 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc8a5c8-b3b4-429d-9410-2ef2d4d5cc67 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:36.661493 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2974346-2d36-4a1d-89b4-70787a30fa48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:36.663218 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2974346-2d36-4a1d-89b4-70787a30fa48 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:36.663642 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2974346-2d36-4a1d-89b4-70787a30fa48 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:36.671297 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2974346-2d36-4a1d-89b4-70787a30fa48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:36.671525 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 434/852] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 21:18:36 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:36.671525 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8a5c8-b3b4-429d-9410-2ef2d4d5cc67 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:36.688303 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2974346-2d36-4a1d-89b4-70787a30fa48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:36.690558 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2974346-2d36-4a1d-89b4-70787a30fa48 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:36.721158 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-e2974346-2d36-4a1d-89b4-70787a30fa48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:36.721158 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:36.721158 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:36.721158 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:36.721158 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:36.721158 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:36.721158 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:36.721158 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:36.721158 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:36.721158 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:36.721158 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:36.721158 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:36.721158 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:36.721158 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:36.721158 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:36.723011 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:36.723011 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:36.723011 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:36.723011 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:36.723011 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:36.723011 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:36.723011 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:36.723011 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:36.723011 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:36.723011 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:36.723011 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:36.723011 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:36.723011 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:36.723011 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:36.723011 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:36.723011 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:36.723011 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:36.724870 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:36.724870 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:36.724870 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:36.724870 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:36.724870 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 419/853] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:36.724870 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2974346-2d36-4a1d-89b4-70787a30fa48 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:36.772268 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:36.807462 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:36.821963 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8a5c8-b3b4-429d-9410-2ef2d4d5cc67 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:36.877898 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a106c9-1f3b-49ee-a7af-ef74ed0225d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:36.878614 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a106c9-1f3b-49ee-a7af-ef74ed0225d6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:36.878895 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a106c9-1f3b-49ee-a7af-ef74ed0225d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:36.879176 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a106c9-1f3b-49ee-a7af-ef74ed0225d6 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:36.879326 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2974346-2d36-4a1d-89b4-70787a30fa48 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:36.879986 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a106c9-1f3b-49ee-a7af-ef74ed0225d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:36.880298 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a106c9-1f3b-49ee-a7af-ef74ed0225d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:36.881886 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a106c9-1f3b-49ee-a7af-ef74ed0225d6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:36.895667 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 435/854] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 21:18:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:36.896584 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a106c9-1f3b-49ee-a7af-ef74ed0225d6 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:36.935075 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b6a64-069d-4932-a5eb-45465c654f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:36.935940 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b6a64-069d-4932-a5eb-45465c654f25 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:36.936442 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b6a64-069d-4932-a5eb-45465c654f25 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:36.936868 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b6a64-069d-4932-a5eb-45465c654f25 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:36.937710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29b6a64-069d-4932-a5eb-45465c654f25 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:36.938209 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29b6a64-069d-4932-a5eb-45465c654f25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:36.940074 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29b6a64-069d-4932-a5eb-45465c654f25 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:36.958281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 420/855] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 21:18:36 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:36.958281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b6a64-069d-4932-a5eb-45465c654f25 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:37.069926 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a106c9-1f3b-49ee-a7af-ef74ed0225d6 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:37.098154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:37.112064 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb73a039-b5a8-4945-a991-9aad179507ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:37.113123 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73a039-b5a8-4945-a991-9aad179507ef None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:37.113123 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73a039-b5a8-4945-a991-9aad179507ef None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:37.113359 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73a039-b5a8-4945-a991-9aad179507ef None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:37.114230 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb73a039-b5a8-4945-a991-9aad179507ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:37.114646 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb73a039-b5a8-4945-a991-9aad179507ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:37.116536 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb73a039-b5a8-4945-a991-9aad179507ef None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:37.137945 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 436/856] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:37.138270 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b6a64-069d-4932-a5eb-45465c654f25 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:37.139922 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb73a039-b5a8-4945-a991-9aad179507ef None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:37.181733 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:37.220443 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdf5a45-0c06-44af-9424-4cc9c68e7999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:37.222018 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf5a45-0c06-44af-9424-4cc9c68e7999 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:37.222813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf5a45-0c06-44af-9424-4cc9c68e7999 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:37.223549 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf5a45-0c06-44af-9424-4cc9c68e7999 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:37.226305 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdf5a45-0c06-44af-9424-4cc9c68e7999 None admin] RBAC: Adding query filter params (role.id=9dc26e65165b49d6916b54458f847755, scope.project.id=c4940befa9c4405699c72deedc8a2127, user.id=eba91dff7cac401e81f6fb6391874af1) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:37.228102 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdf5a45-0c06-44af-9424-4cc9c68e7999 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:37.230975 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdf5a45-0c06-44af-9424-4cc9c68e7999 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:37.244696 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 421/857] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:18:36 2022] GET /identity/v3/role_assignments?user.id=eba91dff7cac401e81f6fb6391874af1&scope.project.id=c4940befa9c4405699c72deedc8a2127&role.id=9dc26e65165b49d6916b54458f847755 => generated 608 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:37.342188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb73a039-b5a8-4945-a991-9aad179507ef None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:37.405477 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ceb628-d02e-424c-b73c-2ba9adb54e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:37.406570 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ceb628-d02e-424c-b73c-2ba9adb54e45 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:37.407036 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ceb628-d02e-424c-b73c-2ba9adb54e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:37.407467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ceb628-d02e-424c-b73c-2ba9adb54e45 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:37.420576 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ceb628-d02e-424c-b73c-2ba9adb54e45 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:37.423560 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ceb628-d02e-424c-b73c-2ba9adb54e45 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:37.451775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-f5ceb628-d02e-424c-b73c-2ba9adb54e45 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 21:18:37.451775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:37.451775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:37.451775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:37.451775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:37.451775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:37.451775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:37.451775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:37.451775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:37.451775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:37.451775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:37.451775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:37.451775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:18:37.451775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:18:37.451775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:18:37.451775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:18:37.452790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:37.452790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:37.452790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:18:37.452790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:37.452790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:18:37.452790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:37.452790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:37.452790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:37.452790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:37.452790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:37.452790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:37.452790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:37.452790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:37.452790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:37.452790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:37.452790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:37.452790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:37.452790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:37.453781 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:37.453781 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:37.453781 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:18:37.453781 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:18:37.453781 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:18:37.453781 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:18:37.453781 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:18:37.453781 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:18:37.453781 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 21:18:37.453781 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:37.455042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 437/858] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 21:18:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:18:37.464468 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdf5a45-0c06-44af-9424-4cc9c68e7999 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:37.571882 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429b3653-18aa-45f7-b1fd-6f3fae4b4455 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:37.572061 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429b3653-18aa-45f7-b1fd-6f3fae4b4455 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:37.572339 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429b3653-18aa-45f7-b1fd-6f3fae4b4455 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:37.574024 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 438/859] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:18:37.580325 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d2818-320f-4f5d-816b-effb7474e030 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:37.580648 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d2818-320f-4f5d-816b-effb7474e030 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:37.581051 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d2818-320f-4f5d-816b-effb7474e030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:37.607691 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e79d2818-320f-4f5d-816b-effb7474e030 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:37.610091 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e79d2818-320f-4f5d-816b-effb7474e030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:37.645150 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdf5a45-0c06-44af-9424-4cc9c68e7999 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:37.685763 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:37.714608 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230fbb06-4155-49d7-9e37-a94c928f73ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:37.715879 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230fbb06-4155-49d7-9e37-a94c928f73ee None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:37.716186 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230fbb06-4155-49d7-9e37-a94c928f73ee None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:37.716439 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230fbb06-4155-49d7-9e37-a94c928f73ee None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:37.717617 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230fbb06-4155-49d7-9e37-a94c928f73ee None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:37.717617 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230fbb06-4155-49d7-9e37-a94c928f73ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:37.719152 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230fbb06-4155-49d7-9e37-a94c928f73ee None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:37.748065 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 422/860] 10.222.0.65 () {56 vars in 1135 bytes} [Thu May 19 21:18:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:37.750878 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 439/861] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:37 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:18:37.757173 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230fbb06-4155-49d7-9e37-a94c928f73ee None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:37.788399 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d2818-320f-4f5d-816b-effb7474e030 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:37.840447 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:37.950360 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230fbb06-4155-49d7-9e37-a94c928f73ee None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:37.952358 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d2818-320f-4f5d-816b-effb7474e030 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:37.992177 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448c6f9c-0516-4537-b029-fdb7e4dc49eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:37.992976 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c6f9c-0516-4537-b029-fdb7e4dc49eb None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:37.993265 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c6f9c-0516-4537-b029-fdb7e4dc49eb None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:37.993541 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c6f9c-0516-4537-b029-fdb7e4dc49eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:38.017583 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3382f60e-d8c6-49a8-bf3d-507fa7376f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:38.018455 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382f60e-d8c6-49a8-bf3d-507fa7376f3e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:38.019078 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382f60e-d8c6-49a8-bf3d-507fa7376f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:38.019566 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382f60e-d8c6-49a8-bf3d-507fa7376f3e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:38.020800 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3382f60e-d8c6-49a8-bf3d-507fa7376f3e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:38.023722 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3382f60e-d8c6-49a8-bf3d-507fa7376f3e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:38.049424 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-3382f60e-d8c6-49a8-bf3d-507fa7376f3e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 21:18:38.049424 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:38.049424 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:18:38.049424 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:18:38.049424 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:18:38.049424 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:18:38.049424 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:18:38.049424 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:18:38.049424 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 21:18:38.049424 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:38.049424 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:18:38.049424 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:38.049424 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:38.049424 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:38.049424 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:38.049424 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:38.050752 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:38.050752 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:38.050752 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:38.050752 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:38.050752 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:38.050752 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:38.050752 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:38.050752 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:18:38.050752 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:18:38.050752 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:18:38.050752 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:18:38.050752 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:38.050752 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:38.050752 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:38.050752 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:38.050752 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:38.052428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:38.052428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:38.052428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:38.052428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:38.052428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:38.052428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:38.052428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:38.052428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:38.052428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:38.052428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:38.052428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:38.052428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:18:38.052428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:18:38.052428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 21:18:38.052428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:38.052428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 440/862] 10.222.0.65 () {56 vars in 1132 bytes} [Thu May 19 21:18:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:38.060493 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3382f60e-d8c6-49a8-bf3d-507fa7376f3e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:38.078731 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448c6f9c-0516-4537-b029-fdb7e4dc49eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:38.080228 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448c6f9c-0516-4537-b029-fdb7e4dc49eb None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:38.129270 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:38.132982 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:38.233157 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3382f60e-d8c6-49a8-bf3d-507fa7376f3e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:38.238937 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 423/863] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:18:37 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 21:18:38.243184 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448c6f9c-0516-4537-b029-fdb7e4dc49eb None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:38.308871 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fef176-c6d0-476b-8588-3fc5416d9aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:38.310669 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fef176-c6d0-476b-8588-3fc5416d9aa9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:38.311473 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fef176-c6d0-476b-8588-3fc5416d9aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:38.312169 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fef176-c6d0-476b-8588-3fc5416d9aa9 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:38.313530 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fef176-c6d0-476b-8588-3fc5416d9aa9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:38.314462 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fef176-c6d0-476b-8588-3fc5416d9aa9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:38.316883 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fef176-c6d0-476b-8588-3fc5416d9aa9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:38.325935 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 441/864] 10.222.0.65 () {56 vars in 1126 bytes} [Thu May 19 21:18:38 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:38.332240 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fef176-c6d0-476b-8588-3fc5416d9aa9 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:38.383267 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448c6f9c-0516-4537-b029-fdb7e4dc49eb None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:38.428261 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc020979-dd74-4be3-8f49-cc4d704d9b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:38.429445 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc020979-dd74-4be3-8f49-cc4d704d9b97 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:38.429909 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc020979-dd74-4be3-8f49-cc4d704d9b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:38.430433 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc020979-dd74-4be3-8f49-cc4d704d9b97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:38.445498 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc020979-dd74-4be3-8f49-cc4d704d9b97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:38.447896 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc020979-dd74-4be3-8f49-cc4d704d9b97 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:38.450711 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:38.487972 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-fc020979-dd74-4be3-8f49-cc4d704d9b97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:38.487972 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:38.487972 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:38.487972 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:38.487972 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:38.487972 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:38.487972 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:38.487972 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:38.487972 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:38.487972 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:38.487972 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:38.487972 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:38.487972 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:38.487972 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:38.487972 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:38.490775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:38.490775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:38.490775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:38.490775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:38.490775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:38.490775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:38.490775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:38.490775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:38.490775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:38.490775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:38.490775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:38.490775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:38.490775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:38.490775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:38.490775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:38.490775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:38.490775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:38.502362 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:38.502362 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:38.502362 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:38.502362 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:38.502362 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 424/865] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:18:38.503432 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc020979-dd74-4be3-8f49-cc4d704d9b97 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:38.543975 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:38.551183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fef176-c6d0-476b-8588-3fc5416d9aa9 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:38.623557 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca21eccc-65b0-41cc-b398-db9e56dedfed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:38.625452 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21eccc-65b0-41cc-b398-db9e56dedfed None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:38.626294 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21eccc-65b0-41cc-b398-db9e56dedfed None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:38.626903 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21eccc-65b0-41cc-b398-db9e56dedfed None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:38.628740 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca21eccc-65b0-41cc-b398-db9e56dedfed None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:38.629426 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca21eccc-65b0-41cc-b398-db9e56dedfed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:38.632115 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca21eccc-65b0-41cc-b398-db9e56dedfed None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:38.642388 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 442/866] 10.222.0.65 () {56 vars in 1126 bytes} [Thu May 19 21:18:38 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:18:38.717311 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc020979-dd74-4be3-8f49-cc4d704d9b97 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:38.791156 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7311de0-a9de-4f8c-a6f8-27c3db4c090c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:38.792360 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7311de0-a9de-4f8c-a6f8-27c3db4c090c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:38.792882 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7311de0-a9de-4f8c-a6f8-27c3db4c090c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:38.793350 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7311de0-a9de-4f8c-a6f8-27c3db4c090c None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:38.794676 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7311de0-a9de-4f8c-a6f8-27c3db4c090c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:38.795198 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7311de0-a9de-4f8c-a6f8-27c3db4c090c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:38.797746 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7311de0-a9de-4f8c-a6f8-27c3db4c090c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:38.822600 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 425/867] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:18:38.828223 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca21eccc-65b0-41cc-b398-db9e56dedfed None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:38.992280 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca21eccc-65b0-41cc-b398-db9e56dedfed None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:39.060947 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a571b8-37a2-4fc9-9175-54fea1183316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:39.062347 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a571b8-37a2-4fc9-9175-54fea1183316 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:39.062818 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a571b8-37a2-4fc9-9175-54fea1183316 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:39.063302 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a571b8-37a2-4fc9-9175-54fea1183316 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:39.077592 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a571b8-37a2-4fc9-9175-54fea1183316 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:39.080497 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a571b8-37a2-4fc9-9175-54fea1183316 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:39.109214 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-03a571b8-37a2-4fc9-9175-54fea1183316 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:39.109214 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:39.109214 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:39.109214 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:39.109214 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:39.109214 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:39.109214 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:39.109214 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:39.109214 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:39.109214 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:39.109214 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:39.109214 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:39.109214 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:18:39.109214 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:18:39.109214 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:18:39.111024 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:18:39.111024 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:39.111024 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:39.111024 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:39.111024 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:39.111024 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:39.111024 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:39.111024 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:39.111024 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:39.111024 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:39.111024 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:39.111024 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:39.111024 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:39.111024 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:39.111024 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:39.111024 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:39.111024 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:39.112817 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:18:39.112817 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:18:39.112817 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:18:39.112817 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:18:39.112817 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:18:39.112817 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:18:39.112817 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:39.112817 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:39.112817 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 443/868] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:18:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:39.124939 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7311de0-a9de-4f8c-a6f8-27c3db4c090c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:39.234693 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:39.324785 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7311de0-a9de-4f8c-a6f8-27c3db4c090c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:39.384923 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0896b4b0-3a59-4bc8-9aee-edacbbf99ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:39.386082 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896b4b0-3a59-4bc8-9aee-edacbbf99ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:39.386585 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896b4b0-3a59-4bc8-9aee-edacbbf99ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:39.387045 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896b4b0-3a59-4bc8-9aee-edacbbf99ca9 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:39.388519 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0896b4b0-3a59-4bc8-9aee-edacbbf99ca9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:39.389045 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0896b4b0-3a59-4bc8-9aee-edacbbf99ca9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:39.391706 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0896b4b0-3a59-4bc8-9aee-edacbbf99ca9 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:39.415926 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 426/869] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 21:18:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:39.417701 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f5df4-e08e-4742-87e0-b2af65789f41 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:39.418373 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f5df4-e08e-4742-87e0-b2af65789f41 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:39.418373 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f5df4-e08e-4742-87e0-b2af65789f41 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:39.419763 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 444/870] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:18:39.426782 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0896b4b0-3a59-4bc8-9aee-edacbbf99ca9 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:39.429004 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509af4ea-50e6-4d54-b572-fb39c5dcea37 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:39.429317 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509af4ea-50e6-4d54-b572-fb39c5dcea37 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:39.429569 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509af4ea-50e6-4d54-b572-fb39c5dcea37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:39.448408 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-509af4ea-50e6-4d54-b572-fb39c5dcea37 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:39.449668 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-509af4ea-50e6-4d54-b572-fb39c5dcea37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:39.471948 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:39.574997 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 445/871] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:39 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:18:39.609034 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509af4ea-50e6-4d54-b572-fb39c5dcea37 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:39.662709 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0896b4b0-3a59-4bc8-9aee-edacbbf99ca9 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:39.698132 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:39.739117 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68efd6e0-5b40-438d-875a-abbd47eb94bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:39.740906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efd6e0-5b40-438d-875a-abbd47eb94bb None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:39.741532 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efd6e0-5b40-438d-875a-abbd47eb94bb None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:39.742109 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efd6e0-5b40-438d-875a-abbd47eb94bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:39.743920 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68efd6e0-5b40-438d-875a-abbd47eb94bb None admin] RBAC: Adding query filter params (role.id=2b714f2e5c6c469f8fa037bda1ed16f6, scope.project.id=c4940befa9c4405699c72deedc8a2127, user.id=7e07984b36934aff9052541ef2a56f36) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:39.744598 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68efd6e0-5b40-438d-875a-abbd47eb94bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:39.747593 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68efd6e0-5b40-438d-875a-abbd47eb94bb None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:39.766463 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 427/872] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:18:39 2022] GET /identity/v3/role_assignments?user.id=7e07984b36934aff9052541ef2a56f36&scope.project.id=c4940befa9c4405699c72deedc8a2127&role.id=2b714f2e5c6c469f8fa037bda1ed16f6 => generated 608 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:39.818561 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509af4ea-50e6-4d54-b572-fb39c5dcea37 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:39.886904 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012005ab-7dac-4e83-b8ba-9452d32a02fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:39.888130 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012005ab-7dac-4e83-b8ba-9452d32a02fd None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:39.888627 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012005ab-7dac-4e83-b8ba-9452d32a02fd None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:39.889110 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012005ab-7dac-4e83-b8ba-9452d32a02fd None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:39.890612 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012005ab-7dac-4e83-b8ba-9452d32a02fd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:39.893344 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012005ab-7dac-4e83-b8ba-9452d32a02fd None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:39.921426 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 446/873] 10.222.0.65 () {60 vars in 1141 bytes} [Thu May 19 21:18:39 2022] POST /identity/v3/services => generated 251 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:18:40.534349 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76583de-1fa9-4efd-b4f5-45bfe054b08e None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:40.535062 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76583de-1fa9-4efd-b4f5-45bfe054b08e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:40.535538 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76583de-1fa9-4efd-b4f5-45bfe054b08e None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:40.537633 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 428/874] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:18:40.543720 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5eb086-7829-44ca-a5b6-b84cb65026cc None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:40.544649 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5eb086-7829-44ca-a5b6-b84cb65026cc None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:40.545143 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5eb086-7829-44ca-a5b6-b84cb65026cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:40.564890 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-4c5eb086-7829-44ca-a5b6-b84cb65026cc None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:40.566587 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4c5eb086-7829-44ca-a5b6-b84cb65026cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:40.579679 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:40.701057 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 447/875] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:40 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:18:40.724179 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76583de-1fa9-4efd-b4f5-45bfe054b08e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:40.931420 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76583de-1fa9-4efd-b4f5-45bfe054b08e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:41.003163 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f19f02-5890-46b8-b9a1-b2635b341497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:41.004415 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f19f02-5890-46b8-b9a1-b2635b341497 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:41.004836 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f19f02-5890-46b8-b9a1-b2635b341497 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:41.005321 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f19f02-5890-46b8-b9a1-b2635b341497 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:41.025141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f19f02-5890-46b8-b9a1-b2635b341497 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:18:41.042586 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-86f19f02-5890-46b8-b9a1-b2635b341497 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 21:18:41.042586 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:41.042586 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:41.042586 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:41.042586 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:41.042586 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:41.042586 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:41.042586 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:41.042586 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:41.042586 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:41.042586 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:41.042586 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:41.042586 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:18:41.042586 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:18:41.042586 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:18:41.042586 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:18:41.044333 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:18:41.044333 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:18:41.044333 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:18:41.044333 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:18:41.044333 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:41.044333 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:41.044333 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:41.044333 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:41.044333 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:41.044333 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:41.044333 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:41.044333 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:41.044333 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:41.044333 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:41.044333 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:41.044333 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:41.045896 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:41.045896 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:41.045896 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:41.045896 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:41.045896 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:18:41.045896 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:18:41.045896 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:18:41.045896 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:18:41.045896 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:18:41.045896 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:18:41.045896 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 21:18:41.045896 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:41.045896 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 429/876] 10.222.0.65 () {56 vars in 1120 bytes} [Thu May 19 21:18:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:18:41.057366 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5eb086-7829-44ca-a5b6-b84cb65026cc None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:41.115605 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da39ed3-919a-40de-8277-52e5ba793afb None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:41.116697 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da39ed3-919a-40de-8277-52e5ba793afb None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:41.117490 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da39ed3-919a-40de-8277-52e5ba793afb None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:41.120860 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 430/877] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:18:41.131500 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097319e4-c212-4e42-9a72-8311563db18b None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:41.132408 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097319e4-c212-4e42-9a72-8311563db18b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:41.133173 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097319e4-c212-4e42-9a72-8311563db18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:41.164632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-097319e4-c212-4e42-9a72-8311563db18b None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:41.167081 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-097319e4-c212-4e42-9a72-8311563db18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:41.233417 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:41.267238 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5eb086-7829-44ca-a5b6-b84cb65026cc None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:41.318294 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 431/878] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:41 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:18:41.352895 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b37db-ed45-4cf6-9b6a-b6802408feab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:41.354181 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b37db-ed45-4cf6-9b6a-b6802408feab None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:41.354729 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b37db-ed45-4cf6-9b6a-b6802408feab None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:41.355209 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b37db-ed45-4cf6-9b6a-b6802408feab None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:41.356571 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045b37db-ed45-4cf6-9b6a-b6802408feab None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:41.357119 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045b37db-ed45-4cf6-9b6a-b6802408feab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:41.359796 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045b37db-ed45-4cf6-9b6a-b6802408feab None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:41.361154 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097319e4-c212-4e42-9a72-8311563db18b None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:41.377685 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 448/879] 10.222.0.65 () {56 vars in 1115 bytes} [Thu May 19 21:18:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:41.386457 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b37db-ed45-4cf6-9b6a-b6802408feab None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:41.391195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:41.575847 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097319e4-c212-4e42-9a72-8311563db18b None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:41.597614 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b37db-ed45-4cf6-9b6a-b6802408feab None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:41.648073 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01525076-089b-4b87-9061-c614f789651b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:41.649081 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01525076-089b-4b87-9061-c614f789651b None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:41.650191 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01525076-089b-4b87-9061-c614f789651b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:41.650919 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01525076-089b-4b87-9061-c614f789651b None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:41.652771 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01525076-089b-4b87-9061-c614f789651b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:41.657384 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01525076-089b-4b87-9061-c614f789651b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:41.677675 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09538dd0-c6db-476e-a96b-7a53dc77b8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:41.679685 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09538dd0-c6db-476e-a96b-7a53dc77b8d9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:41.680554 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09538dd0-c6db-476e-a96b-7a53dc77b8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:41.681213 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09538dd0-c6db-476e-a96b-7a53dc77b8d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:41.691102 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 432/880] 10.222.0.65 () {60 vars in 1141 bytes} [Thu May 19 21:18:41 2022] POST /identity/v3/services => generated 269 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:18:41.708636 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09538dd0-c6db-476e-a96b-7a53dc77b8d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:41.711624 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09538dd0-c6db-476e-a96b-7a53dc77b8d9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:41.749423 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-09538dd0-c6db-476e-a96b-7a53dc77b8d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:41.749423 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:41.749423 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:41.749423 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:41.749423 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:41.749423 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:41.749423 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:41.749423 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:41.749423 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:41.749423 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:41.749423 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:41.749423 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:41.749423 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:41.749423 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:41.749423 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:41.751188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:41.751188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:41.751188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:41.751188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:41.751188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:41.751188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:41.751188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:41.751188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:41.751188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:41.751188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:41.751188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:41.751188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:41.751188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:41.751188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:41.751188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:41.751188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:41.751188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:41.752991 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:41.752991 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:41.752991 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:41.752991 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:41.758093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 449/881] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:18:41.764584 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01525076-089b-4b87-9061-c614f789651b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:41.825027 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:41.976979 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01525076-089b-4b87-9061-c614f789651b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:42.023988 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54906195-6ae9-4474-8adf-d95df7c37509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:42.025152 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54906195-6ae9-4474-8adf-d95df7c37509 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:42.025666 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54906195-6ae9-4474-8adf-d95df7c37509 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:42.025962 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54906195-6ae9-4474-8adf-d95df7c37509 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:42.027160 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54906195-6ae9-4474-8adf-d95df7c37509 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:42.027719 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54906195-6ae9-4474-8adf-d95df7c37509 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:42.030376 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54906195-6ae9-4474-8adf-d95df7c37509 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:42.054451 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 433/882] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:42.063796 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09538dd0-c6db-476e-a96b-7a53dc77b8d9 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:42.133197 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:42.184697 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09538dd0-c6db-476e-a96b-7a53dc77b8d9 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:42.222298 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4772f385-4aad-403c-b628-42c5d085525e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:42.223140 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772f385-4aad-403c-b628-42c5d085525e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:42.223423 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772f385-4aad-403c-b628-42c5d085525e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:42.223700 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772f385-4aad-403c-b628-42c5d085525e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:42.232962 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4772f385-4aad-403c-b628-42c5d085525e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:42.234377 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4772f385-4aad-403c-b628-42c5d085525e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:42.251058 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-4772f385-4aad-403c-b628-42c5d085525e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 21:18:42.251058 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:42.251058 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:42.251058 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:42.251058 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:42.251058 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:42.251058 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:42.251058 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:42.251058 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:42.251058 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:42.251058 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:42.251058 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:42.251058 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:18:42.251058 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:18:42.251058 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:18:42.251058 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:18:42.252069 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:42.252069 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:42.252069 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:18:42.252069 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:42.252069 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:18:42.252069 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:42.252069 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:42.252069 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:42.252069 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:42.252069 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:42.252069 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:42.252069 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:42.252069 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:42.252069 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:42.252069 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:42.252069 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:42.252069 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:42.252069 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:42.253036 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:42.253036 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:42.253036 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:18:42.253036 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:18:42.253036 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:18:42.253036 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:18:42.253036 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:18:42.253036 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:18:42.253036 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 21:18:42.253036 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:42.253036 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 450/883] 10.222.0.65 () {56 vars in 1117 bytes} [Thu May 19 21:18:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:42.259743 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54906195-6ae9-4474-8adf-d95df7c37509 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:42.376843 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:42.398156 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54906195-6ae9-4474-8adf-d95df7c37509 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:42.440582 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a42479d-3682-4507-9c65-ed91ee7b96aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:42.441362 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a42479d-3682-4507-9c65-ed91ee7b96aa None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:42.441648 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a42479d-3682-4507-9c65-ed91ee7b96aa None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:42.441980 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a42479d-3682-4507-9c65-ed91ee7b96aa None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:42.442982 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a42479d-3682-4507-9c65-ed91ee7b96aa None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:42.443356 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a42479d-3682-4507-9c65-ed91ee7b96aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:42.445010 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a42479d-3682-4507-9c65-ed91ee7b96aa None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:42.455311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564e112-f6f6-45f6-b9c0-a726eb42ebd9 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:42.455847 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564e112-f6f6-45f6-b9c0-a726eb42ebd9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:42.456131 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564e112-f6f6-45f6-b9c0-a726eb42ebd9 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:42.459450 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 451/884] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:18:42.466392 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc02c9c-bba9-4cc2-80fe-18b88d1a0f80 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:42.467028 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc02c9c-bba9-4cc2-80fe-18b88d1a0f80 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:42.467509 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc02c9c-bba9-4cc2-80fe-18b88d1a0f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:42.467810 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 434/885] 10.222.0.65 () {56 vars in 1149 bytes} [Thu May 19 21:18:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:42.476402 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a42479d-3682-4507-9c65-ed91ee7b96aa None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:42.488270 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-4dc02c9c-bba9-4cc2-80fe-18b88d1a0f80 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:42.489438 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4dc02c9c-bba9-4cc2-80fe-18b88d1a0f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:42.574319 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 452/886] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:42 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:18:42.595124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc02c9c-bba9-4cc2-80fe-18b88d1a0f80 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:42.615390 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a42479d-3682-4507-9c65-ed91ee7b96aa None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:42.648313 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:42.656657 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a315ed-8171-44df-b607-59baa3ce2da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:42.657183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a315ed-8171-44df-b607-59baa3ce2da9 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:42.657463 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a315ed-8171-44df-b607-59baa3ce2da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:42.657734 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a315ed-8171-44df-b607-59baa3ce2da9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:42.672328 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:42.718923 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc02c9c-bba9-4cc2-80fe-18b88d1a0f80 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:42.741304 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a315ed-8171-44df-b607-59baa3ce2da9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:42.742718 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a315ed-8171-44df-b607-59baa3ce2da9 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:42.761167 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d6f1b6-4ba4-4e85-ae1d-f184e7473faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:42.762453 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6f1b6-4ba4-4e85-ae1d-f184e7473faf None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:42.763059 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6f1b6-4ba4-4e85-ae1d-f184e7473faf None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:42.763544 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6f1b6-4ba4-4e85-ae1d-f184e7473faf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:42.764476 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d6f1b6-4ba4-4e85-ae1d-f184e7473faf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:42.766137 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d6f1b6-4ba4-4e85-ae1d-f184e7473faf None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:42.780443 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-c1d6f1b6-4ba4-4e85-ae1d-f184e7473faf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 21:18:42.780443 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:42.780443 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:18:42.780443 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:18:42.780443 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:18:42.780443 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:18:42.780443 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:18:42.780443 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:18:42.780443 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 21:18:42.780443 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:42.780443 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:18:42.780443 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:42.780443 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:42.780443 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:42.780443 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:42.780443 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:42.780443 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:42.781504 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:42.781504 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:42.781504 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:42.781504 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:42.781504 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:42.781504 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:42.781504 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:18:42.781504 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:18:42.781504 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:18:42.781504 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:18:42.781504 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:42.781504 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:42.781504 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:42.781504 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:42.781504 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:42.781504 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:42.782385 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:42.782385 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:42.782385 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:42.782385 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:42.782385 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:42.782385 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:42.782385 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:42.782385 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:42.782385 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:42.782385 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:42.782385 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:18:42.782385 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:18:42.782385 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 21:18:42.782385 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:42.790410 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 453/887] 10.222.0.65 () {56 vars in 1126 bytes} [Thu May 19 21:18:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:18:42.791793 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d6f1b6-4ba4-4e85-ae1d-f184e7473faf None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:42.874634 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 435/888] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:18:42 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 21:18:42.876476 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a315ed-8171-44df-b607-59baa3ce2da9 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:42.916622 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d6f1b6-4ba4-4e85-ae1d-f184e7473faf None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:42.928236 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:42.930270 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:42.962037 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa9706-e7ee-44bb-8b5f-f762da2fdca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:42.963156 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa9706-e7ee-44bb-8b5f-f762da2fdca7 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:42.963644 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa9706-e7ee-44bb-8b5f-f762da2fdca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:42.964229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa9706-e7ee-44bb-8b5f-f762da2fdca7 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:42.965292 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fa9706-e7ee-44bb-8b5f-f762da2fdca7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:42.965819 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fa9706-e7ee-44bb-8b5f-f762da2fdca7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:42.967464 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fa9706-e7ee-44bb-8b5f-f762da2fdca7 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:42.978820 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 454/889] 10.222.0.65 () {56 vars in 1122 bytes} [Thu May 19 21:18:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:42.980855 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa9706-e7ee-44bb-8b5f-f762da2fdca7 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:43.001135 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a315ed-8171-44df-b607-59baa3ce2da9 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:43.042484 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc8b15-503d-4d4b-b9f4-ef56ad048719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:43.043317 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc8b15-503d-4d4b-b9f4-ef56ad048719 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:43.043595 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc8b15-503d-4d4b-b9f4-ef56ad048719 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:43.043870 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc8b15-503d-4d4b-b9f4-ef56ad048719 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:43.057232 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bc8b15-503d-4d4b-b9f4-ef56ad048719 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:43.059034 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bc8b15-503d-4d4b-b9f4-ef56ad048719 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:43.079761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-c6bc8b15-503d-4d4b-b9f4-ef56ad048719 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:43.079761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:43.079761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:43.079761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:43.079761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:43.079761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:43.079761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:43.079761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:43.079761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:43.079761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:43.079761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:43.079761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:43.079761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:43.079761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:43.079761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:43.080751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:43.080751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:43.080751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:43.080751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:43.080751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:43.080751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:43.080751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:43.080751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:43.080751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:43.080751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:43.080751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:43.080751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:43.080751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:43.080751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:43.080751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:43.080751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:43.080751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:43.081666 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:43.081666 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:43.081666 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:43.081666 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:43.087027 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 436/890] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:43.087808 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc8b15-503d-4d4b-b9f4-ef56ad048719 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:43.110493 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa9706-e7ee-44bb-8b5f-f762da2fdca7 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:43.151996 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4f03e5-a91a-4c5d-a38c-490132f08ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:43.153077 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f03e5-a91a-4c5d-a38c-490132f08ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:43.153701 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f03e5-a91a-4c5d-a38c-490132f08ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:43.154215 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f03e5-a91a-4c5d-a38c-490132f08ac0 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:43.155216 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4f03e5-a91a-4c5d-a38c-490132f08ac0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:43.155751 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4f03e5-a91a-4c5d-a38c-490132f08ac0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:43.157504 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4f03e5-a91a-4c5d-a38c-490132f08ac0 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:43.170702 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 455/891] 10.222.0.65 () {56 vars in 1122 bytes} [Thu May 19 21:18:42 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:18:43.204758 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:43.213614 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc8b15-503d-4d4b-b9f4-ef56ad048719 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:43.260048 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc33002-84ab-4a1a-99e0-858a0c3d1308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:43.261169 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc33002-84ab-4a1a-99e0-858a0c3d1308 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:43.261657 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc33002-84ab-4a1a-99e0-858a0c3d1308 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:43.262150 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc33002-84ab-4a1a-99e0-858a0c3d1308 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:43.263227 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc33002-84ab-4a1a-99e0-858a0c3d1308 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:43.263756 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc33002-84ab-4a1a-99e0-858a0c3d1308 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:43.265521 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc33002-84ab-4a1a-99e0-858a0c3d1308 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:43.279811 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 437/892] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:43.287645 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4f03e5-a91a-4c5d-a38c-490132f08ac0 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:43.322717 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bdc0d0-dd6a-4311-945f-0cdb05d900d3 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:43.323061 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bdc0d0-dd6a-4311-945f-0cdb05d900d3 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:43.323282 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bdc0d0-dd6a-4311-945f-0cdb05d900d3 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:43.324817 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 438/893] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:18:43.335589 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:43.338201 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fcf87-98b4-4e56-8e38-4a4b00b10dce None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:43.338820 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fcf87-98b4-4e56-8e38-4a4b00b10dce None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:43.339306 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09fcf87-98b4-4e56-8e38-4a4b00b10dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:43.373225 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-e09fcf87-98b4-4e56-8e38-4a4b00b10dce None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:43.375465 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e09fcf87-98b4-4e56-8e38-4a4b00b10dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:43.427187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4f03e5-a91a-4c5d-a38c-490132f08ac0 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:43.471575 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922431f-378a-43c8-9372-2c86e69c2824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:43.472708 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922431f-378a-43c8-9372-2c86e69c2824 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:43.473161 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922431f-378a-43c8-9372-2c86e69c2824 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:43.473629 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922431f-378a-43c8-9372-2c86e69c2824 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:43.482333 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b922431f-378a-43c8-9372-2c86e69c2824 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:43.484097 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b922431f-378a-43c8-9372-2c86e69c2824 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:43.502030 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-b922431f-378a-43c8-9372-2c86e69c2824 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:43.502030 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:43.502030 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:43.502030 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:43.502030 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:43.502030 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:43.502030 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:43.502030 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:43.502030 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:43.502030 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:43.502030 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:43.502030 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:43.502030 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:18:43.502030 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:18:43.502030 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:18:43.503101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:18:43.503101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:43.503101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:43.503101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:43.503101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:43.503101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:43.503101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:43.503101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:43.503101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:43.503101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:43.503101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:43.503101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:43.503101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:43.503101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:43.503101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:43.503101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:43.503101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:43.504013 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:18:43.504013 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:18:43.504013 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:18:43.504013 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:18:43.504013 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:18:43.504013 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:18:43.504013 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:43.504013 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:43.510647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 456/894] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:18:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:43.513405 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922431f-378a-43c8-9372-2c86e69c2824 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:43.518351 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 439/895] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:43 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:18:43.543105 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09fcf87-98b4-4e56-8e38-4a4b00b10dce None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:43.653377 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:43.668295 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:43.676270 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922431f-378a-43c8-9372-2c86e69c2824 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:43.677239 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09fcf87-98b4-4e56-8e38-4a4b00b10dce None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:43.719651 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f1225-4b70-499f-8d1f-e16b6a39ee1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:43.720710 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f1225-4b70-499f-8d1f-e16b6a39ee1c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:43.721154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f1225-4b70-499f-8d1f-e16b6a39ee1c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:43.721656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f1225-4b70-499f-8d1f-e16b6a39ee1c None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:43.722821 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179f1225-4b70-499f-8d1f-e16b6a39ee1c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:43.723313 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179f1225-4b70-499f-8d1f-e16b6a39ee1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:43.725342 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179f1225-4b70-499f-8d1f-e16b6a39ee1c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:43.731655 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53907bbb-b35f-4962-9d5a-890d089916eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:43.732969 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53907bbb-b35f-4962-9d5a-890d089916eb None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:43.733345 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53907bbb-b35f-4962-9d5a-890d089916eb None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:43.733712 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53907bbb-b35f-4962-9d5a-890d089916eb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:43.734536 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53907bbb-b35f-4962-9d5a-890d089916eb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:43.736826 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53907bbb-b35f-4962-9d5a-890d089916eb None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:43.747913 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 457/896] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 21:18:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:43.749205 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f1225-4b70-499f-8d1f-e16b6a39ee1c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:43.763280 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-53907bbb-b35f-4962-9d5a-890d089916eb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 21:18:43.763280 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:43.763280 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:18:43.763280 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:18:43.763280 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:18:43.763280 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:18:43.763280 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:18:43.763280 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:18:43.763280 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 21:18:43.763280 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:43.763280 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:18:43.763280 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:43.763280 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:43.763280 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:43.763280 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:43.763280 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:43.763280 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:43.764415 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:43.764415 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:43.764415 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:43.764415 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:43.764415 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:43.764415 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:43.764415 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:18:43.764415 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:18:43.764415 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:18:43.764415 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:18:43.764415 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:43.764415 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:43.764415 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:43.764415 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:43.764415 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:43.764415 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:43.765363 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:43.765363 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:43.765363 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:43.765363 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:43.765363 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:43.765363 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:43.765363 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:43.765363 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:43.765363 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:43.765363 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:43.765363 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:18:43.765363 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:18:43.765363 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 21:18:43.765363 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:43.768665 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 440/897] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 21:18:43 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:18:43.778987 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53907bbb-b35f-4962-9d5a-890d089916eb None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:43.905411 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f1225-4b70-499f-8d1f-e16b6a39ee1c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:43.945163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d258600-e7b1-4571-9920-cd9f2b21e760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:43.945765 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d258600-e7b1-4571-9920-cd9f2b21e760 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:43.946044 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d258600-e7b1-4571-9920-cd9f2b21e760 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:43.946362 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d258600-e7b1-4571-9920-cd9f2b21e760 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:43.947213 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d258600-e7b1-4571-9920-cd9f2b21e760 None admin] RBAC: Adding query filter params (role.id=bdd2ca99a0d74237b4642eae68a9a1c5, scope.project.id=c4940befa9c4405699c72deedc8a2127, user.id=eba91dff7cac401e81f6fb6391874af1) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:43.947525 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d258600-e7b1-4571-9920-cd9f2b21e760 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:43.949033 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d258600-e7b1-4571-9920-cd9f2b21e760 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:43.960835 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 458/898] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:18:43 2022] GET /identity/v3/role_assignments?user.id=eba91dff7cac401e81f6fb6391874af1&scope.project.id=c4940befa9c4405699c72deedc8a2127&role.id=bdd2ca99a0d74237b4642eae68a9a1c5 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:44.008773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53907bbb-b35f-4962-9d5a-890d089916eb None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:44.080853 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df9c546-d856-4fa4-acd6-4b12dd046406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:44.082605 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9c546-d856-4fa4-acd6-4b12dd046406 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:44.083585 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9c546-d856-4fa4-acd6-4b12dd046406 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:44.084517 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9c546-d856-4fa4-acd6-4b12dd046406 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:44.086363 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df9c546-d856-4fa4-acd6-4b12dd046406 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:44.087284 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df9c546-d856-4fa4-acd6-4b12dd046406 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:44.090363 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df9c546-d856-4fa4-acd6-4b12dd046406 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:44.106425 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 441/899] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 21:18:43 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:44.111451 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d258600-e7b1-4571-9920-cd9f2b21e760 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:44.163169 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:44.308675 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d258600-e7b1-4571-9920-cd9f2b21e760 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:44.351843 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85481476-515f-4525-9f12-b3691408ea70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:44.352526 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85481476-515f-4525-9f12-b3691408ea70 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:44.352802 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85481476-515f-4525-9f12-b3691408ea70 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:44.353073 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85481476-515f-4525-9f12-b3691408ea70 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:44.353784 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85481476-515f-4525-9f12-b3691408ea70 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:44.354085 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85481476-515f-4525-9f12-b3691408ea70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:44.355491 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85481476-515f-4525-9f12-b3691408ea70 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:44.361731 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 459/900] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 21:18:44 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:44.371845 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df9c546-d856-4fa4-acd6-4b12dd046406 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:44.400102 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:44.541165 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df9c546-d856-4fa4-acd6-4b12dd046406 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:44.583714 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badf1052-dce8-479c-ae63-47234913d2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:44.584780 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf1052-dce8-479c-ae63-47234913d2af None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:44.584780 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf1052-dce8-479c-ae63-47234913d2af None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:44.585101 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf1052-dce8-479c-ae63-47234913d2af None admin] PATH_INFO: `/v3/endpoints` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:44.585916 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badf1052-dce8-479c-ae63-47234913d2af None admin] RBAC: Adding query filter params (interface=public, service_id=97091fe2d90e472794c1172c78485a59, region_id=RegionOne) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:44.586228 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badf1052-dce8-479c-ae63-47234913d2af None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:44.587873 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badf1052-dce8-479c-ae63-47234913d2af None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:44.595584 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 442/901] 10.222.0.65 () {56 vars in 1251 bytes} [Thu May 19 21:18:44 2022] GET /identity/v3/endpoints?service_id=97091fe2d90e472794c1172c78485a59&interface=public®ion_id=RegionOne => generated 194 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:44.605772 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85481476-515f-4525-9f12-b3691408ea70 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:44.701936 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:44.722494 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85481476-515f-4525-9f12-b3691408ea70 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:44.765197 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1bff50-449e-40ed-befc-82549041c9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:44.765900 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1bff50-449e-40ed-befc-82549041c9dd None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:44.766186 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1bff50-449e-40ed-befc-82549041c9dd None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:44.766499 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1bff50-449e-40ed-befc-82549041c9dd None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:44.767174 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1bff50-449e-40ed-befc-82549041c9dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:44.768615 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1bff50-449e-40ed-befc-82549041c9dd None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:44.776223 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 460/902] 10.222.0.65 () {56 vars in 1087 bytes} [Thu May 19 21:18:44 2022] GET /identity/v3/services => generated 1268 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 21:18:44.831574 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dab990-fc79-448a-bf0c-e5fbad4430a5 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:44.832018 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dab990-fc79-448a-bf0c-e5fbad4430a5 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:44.832473 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dab990-fc79-448a-bf0c-e5fbad4430a5 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:44.835490 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 443/903] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:18:44.844125 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b0504-a539-4652-950c-e4ec30127119 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:44.844594 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b0504-a539-4652-950c-e4ec30127119 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:44.845025 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b0504-a539-4652-950c-e4ec30127119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:44.873925 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-425b0504-a539-4652-950c-e4ec30127119 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:44.875109 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-425b0504-a539-4652-950c-e4ec30127119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:44.957345 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 461/904] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:44 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:18:44.993890 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dab990-fc79-448a-bf0c-e5fbad4430a5 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:45.122181 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:45.154832 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dab990-fc79-448a-bf0c-e5fbad4430a5 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:45.194762 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b6045-fa1f-48ee-93de-69ac5fce61b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:45.195637 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b6045-fa1f-48ee-93de-69ac5fce61b7 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:45.195921 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b6045-fa1f-48ee-93de-69ac5fce61b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:45.196198 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b6045-fa1f-48ee-93de-69ac5fce61b7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:45.196862 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996b6045-fa1f-48ee-93de-69ac5fce61b7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:45.198475 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996b6045-fa1f-48ee-93de-69ac5fce61b7 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:45.212774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-996b6045-fa1f-48ee-93de-69ac5fce61b7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 21:18:45.212774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:45.212774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:18:45.212774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:18:45.212774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:18:45.212774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:18:45.212774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:18:45.212774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:18:45.212774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 21:18:45.212774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:45.212774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:18:45.212774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:45.212774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:45.212774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:45.212774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:45.212774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:45.213805 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:45.213805 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:45.213805 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:45.213805 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:45.213805 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:45.213805 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:45.213805 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:45.213805 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:18:45.213805 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:18:45.213805 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:18:45.213805 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:18:45.213805 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:45.213805 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:45.213805 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:45.213805 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:45.213805 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:45.214710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:45.214710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:45.214710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:45.214710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:45.214710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:45.214710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:45.214710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:45.214710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:45.214710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:45.214710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:45.214710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:45.214710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:18:45.214710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:18:45.214710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 21:18:45.214710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:45.214710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 444/905] 10.222.0.65 () {56 vars in 1132 bytes} [Thu May 19 21:18:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:45.221992 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b0504-a539-4652-950c-e4ec30127119 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:45.304451 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:45.328678 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b0504-a539-4652-950c-e4ec30127119 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:45.370089 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5432a942-cdb4-4431-b023-e2b1d1887fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:45.370837 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5432a942-cdb4-4431-b023-e2b1d1887fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:45.371115 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5432a942-cdb4-4431-b023-e2b1d1887fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:45.371388 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5432a942-cdb4-4431-b023-e2b1d1887fb3 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:45.372123 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5432a942-cdb4-4431-b023-e2b1d1887fb3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:45.372428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5432a942-cdb4-4431-b023-e2b1d1887fb3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:45.373860 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5432a942-cdb4-4431-b023-e2b1d1887fb3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:45.380063 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 462/906] 10.222.0.65 () {56 vars in 1126 bytes} [Thu May 19 21:18:45 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:45.386023 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b6045-fa1f-48ee-93de-69ac5fce61b7 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:45.499359 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b6045-fa1f-48ee-93de-69ac5fce61b7 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:45.560875 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9bd498-1a51-4e4e-b463-7a9fa33761bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:45.566280 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9bd498-1a51-4e4e-b463-7a9fa33761bb None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:45.566280 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9bd498-1a51-4e4e-b463-7a9fa33761bb None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:45.566280 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9bd498-1a51-4e4e-b463-7a9fa33761bb None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:45.566280 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9bd498-1a51-4e4e-b463-7a9fa33761bb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:45.566280 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9bd498-1a51-4e4e-b463-7a9fa33761bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:45.567025 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9bd498-1a51-4e4e-b463-7a9fa33761bb None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:45.575252 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 445/907] 10.222.0.65 () {56 vars in 1126 bytes} [Thu May 19 21:18:45 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:45.582590 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5432a942-cdb4-4431-b023-e2b1d1887fb3 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:45.740137 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5432a942-cdb4-4431-b023-e2b1d1887fb3 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:45.785127 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06643d5e-ca26-4149-afcc-51a8e0cdae86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:45.786161 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06643d5e-ca26-4149-afcc-51a8e0cdae86 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:45.786976 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06643d5e-ca26-4149-afcc-51a8e0cdae86 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:45.787310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06643d5e-ca26-4149-afcc-51a8e0cdae86 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:45.788970 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06643d5e-ca26-4149-afcc-51a8e0cdae86 None admin] RBAC: Adding query filter params (interface=public, service_id=7d589eb357984088a41b79da5c273f5f, region_id=RegionOne) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:45.789409 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06643d5e-ca26-4149-afcc-51a8e0cdae86 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:45.791938 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06643d5e-ca26-4149-afcc-51a8e0cdae86 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:45.807447 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 463/908] 10.222.0.65 () {56 vars in 1251 bytes} [Thu May 19 21:18:45 2022] GET /identity/v3/endpoints?service_id=7d589eb357984088a41b79da5c273f5f&interface=public®ion_id=RegionOne => generated 194 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:45.816654 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9bd498-1a51-4e4e-b463-7a9fa33761bb None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:45.823188 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:45.836905 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344c9e4-cf7e-41f6-b349-929e4fce6420 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:45.837406 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344c9e4-cf7e-41f6-b349-929e4fce6420 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:45.837912 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344c9e4-cf7e-41f6-b349-929e4fce6420 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:45.841742 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 464/909] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:18:45.853577 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89adc52-e183-4b05-a5bc-692e5af0f55d None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:45.853577 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89adc52-e183-4b05-a5bc-692e5af0f55d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:45.853945 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89adc52-e183-4b05-a5bc-692e5af0f55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:45.875757 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:45.886187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b89adc52-e183-4b05-a5bc-692e5af0f55d None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:45.888840 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b89adc52-e183-4b05-a5bc-692e5af0f55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:45.966364 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9bd498-1a51-4e4e-b463-7a9fa33761bb None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:45.997690 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 465/910] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:45 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:18:46.020187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89adc52-e183-4b05-a5bc-692e5af0f55d None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:46.021103 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b77df6d-b6cb-4d0b-98e6-99a6ea0ccf11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:46.022009 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77df6d-b6cb-4d0b-98e6-99a6ea0ccf11 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:46.022438 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77df6d-b6cb-4d0b-98e6-99a6ea0ccf11 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:46.022817 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77df6d-b6cb-4d0b-98e6-99a6ea0ccf11 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:46.023618 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b77df6d-b6cb-4d0b-98e6-99a6ea0ccf11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:46.025021 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b77df6d-b6cb-4d0b-98e6-99a6ea0ccf11 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:46.037312 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 446/911] 10.222.0.65 () {56 vars in 1087 bytes} [Thu May 19 21:18:45 2022] GET /identity/v3/services => generated 1268 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:18:46.145086 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89adc52-e183-4b05-a5bc-692e5af0f55d None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:46.184703 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a56d0-65ef-4074-9125-f7cd473dc548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:46.185459 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a56d0-65ef-4074-9125-f7cd473dc548 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:46.185744 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a56d0-65ef-4074-9125-f7cd473dc548 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:46.186022 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a56d0-65ef-4074-9125-f7cd473dc548 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:46.186831 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06a56d0-65ef-4074-9125-f7cd473dc548 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:46.188267 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06a56d0-65ef-4074-9125-f7cd473dc548 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:46.191960 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:46.207895 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 466/912] 10.222.0.65 () {60 vars in 1141 bytes} [Thu May 19 21:18:46 2022] POST /identity/v3/services => generated 241 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:18:46.806427 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad68e0-c6bb-459d-bf77-4d71671883fc None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:46.807126 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad68e0-c6bb-459d-bf77-4d71671883fc None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:46.807700 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad68e0-c6bb-459d-bf77-4d71671883fc None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:46.810897 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 447/913] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:18:46.820272 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aab78c-3990-4add-99df-692d0460c273 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:46.820929 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aab78c-3990-4add-99df-692d0460c273 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:46.821567 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aab78c-3990-4add-99df-692d0460c273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:46.858591 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-69aab78c-3990-4add-99df-692d0460c273 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:46.858591 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-69aab78c-3990-4add-99df-692d0460c273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:47.013217 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 467/914] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:18:47.050542 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cad68e0-c6bb-459d-bf77-4d71671883fc None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:47.185541 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:47.252980 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cad68e0-c6bb-459d-bf77-4d71671883fc None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:47.320682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4010db8e-6c49-450b-a507-cda50030a486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:47.322598 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010db8e-6c49-450b-a507-cda50030a486 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:47.323399 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010db8e-6c49-450b-a507-cda50030a486 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:47.324179 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010db8e-6c49-450b-a507-cda50030a486 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:47.343863 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4010db8e-6c49-450b-a507-cda50030a486 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:18:47.359416 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-4010db8e-6c49-450b-a507-cda50030a486 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 21:18:47.359416 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:47.359416 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:47.359416 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:47.359416 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:47.359416 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:47.359416 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:47.359416 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:47.359416 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:47.359416 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:47.359416 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:47.359416 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:47.359416 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:18:47.359416 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:18:47.359416 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:18:47.359416 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:18:47.361194 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:18:47.361194 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:18:47.361194 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:18:47.361194 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:18:47.361194 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:47.361194 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:47.361194 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:47.361194 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:47.361194 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:47.361194 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:47.361194 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:47.361194 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:47.361194 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:47.361194 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:47.361194 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:47.361194 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:47.363382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:47.363382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:47.363382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:47.363382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:47.363382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:18:47.363382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:18:47.363382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:18:47.363382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:18:47.363382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:18:47.363382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:18:47.363382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 21:18:47.363382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:47.368229 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 448/915] 10.222.0.65 () {56 vars in 1120 bytes} [Thu May 19 21:18:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:47.376375 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aab78c-3990-4add-99df-692d0460c273 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:47.447148 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ad244-ce65-4254-9f21-17b771de641f None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:47.448005 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ad244-ce65-4254-9f21-17b771de641f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:47.448710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ad244-ce65-4254-9f21-17b771de641f None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:47.451691 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 449/916] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:18:47.458994 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab5582-517c-4e1d-a2da-b61b94f1cec2 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:47.458994 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab5582-517c-4e1d-a2da-b61b94f1cec2 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:47.458994 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab5582-517c-4e1d-a2da-b61b94f1cec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:47.476011 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-cbab5582-517c-4e1d-a2da-b61b94f1cec2 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:47.477276 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-cbab5582-517c-4e1d-a2da-b61b94f1cec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:47.566597 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 450/917] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:18:47.571664 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aab78c-3990-4add-99df-692d0460c273 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:47.581927 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab5582-517c-4e1d-a2da-b61b94f1cec2 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:47.621622 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:47.644931 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f3bd70-1b5a-4a74-8317-6294bfed416c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:47.646059 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f3bd70-1b5a-4a74-8317-6294bfed416c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:47.646534 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f3bd70-1b5a-4a74-8317-6294bfed416c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:47.646982 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f3bd70-1b5a-4a74-8317-6294bfed416c None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:47.648283 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f3bd70-1b5a-4a74-8317-6294bfed416c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:47.648909 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f3bd70-1b5a-4a74-8317-6294bfed416c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:47.651423 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f3bd70-1b5a-4a74-8317-6294bfed416c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:47.675055 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:47.675409 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 468/918] 10.222.0.65 () {56 vars in 1115 bytes} [Thu May 19 21:18:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:47.678672 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f3bd70-1b5a-4a74-8317-6294bfed416c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:47.760810 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab5582-517c-4e1d-a2da-b61b94f1cec2 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:47.821323 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ccc85-0f02-4965-891f-11f08557cbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:47.823745 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ccc85-0f02-4965-891f-11f08557cbf5 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:47.824377 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ccc85-0f02-4965-891f-11f08557cbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:47.825005 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ccc85-0f02-4965-891f-11f08557cbf5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:47.826381 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3ccc85-0f02-4965-891f-11f08557cbf5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:47.828945 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3ccc85-0f02-4965-891f-11f08557cbf5 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:47.852806 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-dc3ccc85-0f02-4965-891f-11f08557cbf5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 21:18:47.852806 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:47.852806 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:18:47.852806 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:18:47.852806 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:18:47.852806 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:18:47.852806 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:18:47.852806 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:18:47.852806 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 21:18:47.852806 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:47.852806 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:18:47.852806 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:47.852806 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:47.852806 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:47.852806 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:47.852806 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:47.852806 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:47.854694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:47.854694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:47.854694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:47.854694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:47.854694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:47.854694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:47.854694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:18:47.854694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:18:47.854694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:18:47.854694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:18:47.854694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:47.854694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:47.854694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:47.854694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:47.854694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:47.854694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:47.856043 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:47.856043 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:47.856043 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:47.856043 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:47.856043 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:47.856043 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:47.856043 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:47.856043 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:47.856043 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:47.856043 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:47.856043 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:18:47.856043 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:18:47.856043 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 21:18:47.856043 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:47.860450 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 451/919] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 21:18:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:47.869626 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ccc85-0f02-4965-891f-11f08557cbf5 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:47.898415 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f3bd70-1b5a-4a74-8317-6294bfed416c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:47.975117 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0f7f7-3618-45f9-b35c-eac60e79e555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:47.976481 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0f7f7-3618-45f9-b35c-eac60e79e555 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:47.976974 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0f7f7-3618-45f9-b35c-eac60e79e555 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:47.977458 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0f7f7-3618-45f9-b35c-eac60e79e555 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:47.985913 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ccc85-0f02-4965-891f-11f08557cbf5 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:47.998482 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c0f7f7-3618-45f9-b35c-eac60e79e555 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:48.001162 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c0f7f7-3618-45f9-b35c-eac60e79e555 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:48.012512 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:48.026696 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d5680b-6eb5-4c8c-810f-68f41800b9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:48.027671 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5680b-6eb5-4c8c-810f-68f41800b9a3 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:48.028095 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5680b-6eb5-4c8c-810f-68f41800b9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:48.028550 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5680b-6eb5-4c8c-810f-68f41800b9a3 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:48.029437 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d5680b-6eb5-4c8c-810f-68f41800b9a3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:48.029884 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d5680b-6eb5-4c8c-810f-68f41800b9a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:48.031475 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d5680b-6eb5-4c8c-810f-68f41800b9a3 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:48.035733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-12c0f7f7-3618-45f9-b35c-eac60e79e555 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:48.035733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:48.035733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:48.035733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:48.035733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:48.035733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:48.035733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:48.035733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:48.035733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:48.035733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:48.035733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:48.035733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:48.035733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:48.035733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:48.035733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:48.036965 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:48.036965 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:48.036965 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:48.036965 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:48.036965 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:48.036965 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:48.036965 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:48.036965 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:48.036965 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:48.036965 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:48.036965 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:48.036965 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:48.036965 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:48.036965 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:48.036965 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:48.036965 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:48.036965 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:48.038010 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:48.038010 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:48.038010 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:48.038010 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:48.038359 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 469/920] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:48.045246 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 452/921] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 21:18:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:48.047390 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0f7f7-3618-45f9-b35c-eac60e79e555 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:48.048087 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d5680b-6eb5-4c8c-810f-68f41800b9a3 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:48.116463 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:48.200459 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0f7f7-3618-45f9-b35c-eac60e79e555 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:48.241362 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece18cf-374d-4850-9eb9-0b279924d7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:48.242496 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece18cf-374d-4850-9eb9-0b279924d7bf None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:48.243000 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece18cf-374d-4850-9eb9-0b279924d7bf None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:48.243475 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece18cf-374d-4850-9eb9-0b279924d7bf None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:48.244398 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d5680b-6eb5-4c8c-810f-68f41800b9a3 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:48.244649 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aece18cf-374d-4850-9eb9-0b279924d7bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:48.245146 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aece18cf-374d-4850-9eb9-0b279924d7bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:48.246800 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aece18cf-374d-4850-9eb9-0b279924d7bf None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:48.262118 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 470/922] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:48.270095 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece18cf-374d-4850-9eb9-0b279924d7bf None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:48.316247 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97911eb3-6046-41ae-9634-d48789a850e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:48.317427 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97911eb3-6046-41ae-9634-d48789a850e6 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:48.318438 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97911eb3-6046-41ae-9634-d48789a850e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:48.318908 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97911eb3-6046-41ae-9634-d48789a850e6 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:48.320261 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97911eb3-6046-41ae-9634-d48789a850e6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:48.320793 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97911eb3-6046-41ae-9634-d48789a850e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:48.323404 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97911eb3-6046-41ae-9634-d48789a850e6 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:48.339069 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 453/923] 10.222.0.65 () {56 vars in 1114 bytes} [Thu May 19 21:18:48 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:48.343006 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97911eb3-6046-41ae-9634-d48789a850e6 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:48.393801 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:48.403124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece18cf-374d-4850-9eb9-0b279924d7bf None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:48.442050 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:48.443676 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c74528f-e1b0-4d80-8cb9-704ecbff23a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:48.444831 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74528f-e1b0-4d80-8cb9-704ecbff23a1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:48.445328 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74528f-e1b0-4d80-8cb9-704ecbff23a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:48.445832 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74528f-e1b0-4d80-8cb9-704ecbff23a1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:48.457043 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c74528f-e1b0-4d80-8cb9-704ecbff23a1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:48.459276 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c74528f-e1b0-4d80-8cb9-704ecbff23a1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:48.477185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-7c74528f-e1b0-4d80-8cb9-704ecbff23a1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 21:18:48.477185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:48.477185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:48.477185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:48.477185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:48.477185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:48.477185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:48.477185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:48.477185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:48.477185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:48.477185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:48.477185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:48.477185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:18:48.477185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:18:48.477185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:18:48.477185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:18:48.478230 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:48.478230 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:48.478230 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:18:48.478230 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:48.478230 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:18:48.478230 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:48.478230 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:48.478230 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:48.478230 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:48.478230 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:48.478230 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:48.478230 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:48.478230 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:48.478230 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:48.478230 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:48.478230 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:48.478230 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:48.478230 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:48.479242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:48.479242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:48.479242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:18:48.479242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:18:48.479242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:18:48.479242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:18:48.479242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:18:48.479242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:18:48.479242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 21:18:48.479242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:48.482652 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 471/924] 10.222.0.65 () {56 vars in 1117 bytes} [Thu May 19 21:18:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:48.491125 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c74528f-e1b0-4d80-8cb9-704ecbff23a1 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:48.551155 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97911eb3-6046-41ae-9634-d48789a850e6 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:48.609570 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c74528f-e1b0-4d80-8cb9-704ecbff23a1 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:48.619941 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beaac54-418c-40d3-aa91-270a84f241fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:48.621079 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beaac54-418c-40d3-aa91-270a84f241fe None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:48.621566 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beaac54-418c-40d3-aa91-270a84f241fe None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:48.622355 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beaac54-418c-40d3-aa91-270a84f241fe None admin] PATH_INFO: `/v3/endpoints` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:48.623505 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beaac54-418c-40d3-aa91-270a84f241fe None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:48.626362 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beaac54-418c-40d3-aa91-270a84f241fe None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:48.653610 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54248537-8b50-440b-aa3e-d1938255f8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:48.654787 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54248537-8b50-440b-aa3e-d1938255f8ba None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:48.655284 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54248537-8b50-440b-aa3e-d1938255f8ba None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:48.656103 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54248537-8b50-440b-aa3e-d1938255f8ba None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:48.657267 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54248537-8b50-440b-aa3e-d1938255f8ba None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:48.657869 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54248537-8b50-440b-aa3e-d1938255f8ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:48.659549 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54248537-8b50-440b-aa3e-d1938255f8ba None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:48.674272 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 454/925] 10.222.0.65 () {60 vars in 1144 bytes} [Thu May 19 21:18:48 2022] POST /identity/v3/endpoints => generated 346 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:18:48.701430 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 472/926] 10.222.0.65 () {56 vars in 1149 bytes} [Thu May 19 21:18:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:48.711004 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beaac54-418c-40d3-aa91-270a84f241fe None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:48.767519 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc34dea-9555-4e44-9b13-527c96966c8c None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:48.767843 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc34dea-9555-4e44-9b13-527c96966c8c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:48.768077 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc34dea-9555-4e44-9b13-527c96966c8c None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:48.770703 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 473/927] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:18:48.776597 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9581f5-e71f-47b9-acbc-47beaf19349e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:48.776946 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9581f5-e71f-47b9-acbc-47beaf19349e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:48.777161 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9581f5-e71f-47b9-acbc-47beaf19349e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:48.803667 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-2f9581f5-e71f-47b9-acbc-47beaf19349e None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:48.805771 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2f9581f5-e71f-47b9-acbc-47beaf19349e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:48.828605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:48.863208 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beaac54-418c-40d3-aa91-270a84f241fe None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:48.866526 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:48.911385 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f530b-93f7-4a5c-97a4-fe5992df9056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:48.912717 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f530b-93f7-4a5c-97a4-fe5992df9056 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:48.913151 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f530b-93f7-4a5c-97a4-fe5992df9056 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:48.913604 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f530b-93f7-4a5c-97a4-fe5992df9056 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:48.925966 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040f530b-93f7-4a5c-97a4-fe5992df9056 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:48.929985 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040f530b-93f7-4a5c-97a4-fe5992df9056 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:48.948868 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-040f530b-93f7-4a5c-97a4-fe5992df9056 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:48.948868 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:48.948868 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:48.948868 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:48.948868 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:48.948868 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:48.948868 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:48.948868 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:48.948868 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:48.948868 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:48.948868 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:48.948868 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:48.948868 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:48.948868 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:48.948868 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:48.950059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:48.950059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:48.950059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:48.950059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:48.950059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:48.950059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:48.950059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:48.950059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:48.950059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:48.950059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:48.950059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:48.950059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:48.950059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:48.950059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:48.950059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:48.950059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:48.950059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:48.951203 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:48.951203 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:48.951203 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:48.951203 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:48.953301 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 455/928] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:48.955083 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 474/929] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:18:48.966261 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f530b-93f7-4a5c-97a4-fe5992df9056 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:48.979647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9581f5-e71f-47b9-acbc-47beaf19349e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:49.128074 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f530b-93f7-4a5c-97a4-fe5992df9056 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:49.165301 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9581f5-e71f-47b9-acbc-47beaf19349e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:49.170407 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f133e81e-bed2-47e9-826a-7e00b6af8e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:49.171360 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133e81e-bed2-47e9-826a-7e00b6af8e76 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:49.171869 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133e81e-bed2-47e9-826a-7e00b6af8e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:49.172427 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133e81e-bed2-47e9-826a-7e00b6af8e76 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:49.173408 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f133e81e-bed2-47e9-826a-7e00b6af8e76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:49.173973 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f133e81e-bed2-47e9-826a-7e00b6af8e76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:49.175615 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f133e81e-bed2-47e9-826a-7e00b6af8e76 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:49.198319 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 456/930] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:49.203131 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f133e81e-bed2-47e9-826a-7e00b6af8e76 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:49.224160 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efab89c-91b2-4b96-aef9-7e0fd59df0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:49.225387 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efab89c-91b2-4b96-aef9-7e0fd59df0c2 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:49.225387 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efab89c-91b2-4b96-aef9-7e0fd59df0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:49.225656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efab89c-91b2-4b96-aef9-7e0fd59df0c2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:49.226317 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efab89c-91b2-4b96-aef9-7e0fd59df0c2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:49.227789 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efab89c-91b2-4b96-aef9-7e0fd59df0c2 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:49.240345 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:49.242200 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-3efab89c-91b2-4b96-aef9-7e0fd59df0c2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 21:18:49.242200 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:49.242200 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:18:49.242200 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:18:49.242200 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:18:49.242200 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:18:49.242200 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:18:49.242200 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:18:49.242200 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 21:18:49.242200 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:49.242200 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:18:49.242200 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:49.242200 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:49.242200 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:49.242200 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:49.242200 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:49.245301 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:49.245301 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:49.245301 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:49.245301 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:49.245301 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:49.245301 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:49.245301 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:49.245301 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:18:49.245301 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:18:49.245301 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:18:49.245301 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:18:49.245301 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:49.245301 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:49.245301 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:49.245301 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:49.245301 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:49.247139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:49.247139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:49.247139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:49.247139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:49.247139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:49.247139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:49.247139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:49.247139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:49.247139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:49.247139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:49.247139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:49.247139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:18:49.247139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:18:49.247139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 21:18:49.247139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:49.247139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 475/931] 10.222.0.65 () {56 vars in 1132 bytes} [Thu May 19 21:18:48 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:49.249815 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efab89c-91b2-4b96-aef9-7e0fd59df0c2 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:49.267162 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:49.370866 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f133e81e-bed2-47e9-826a-7e00b6af8e76 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:49.401083 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efab89c-91b2-4b96-aef9-7e0fd59df0c2 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:49.415846 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142eab21-a3a5-4b1a-8657-d23c08c25edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:49.416817 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142eab21-a3a5-4b1a-8657-d23c08c25edd None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:49.417499 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142eab21-a3a5-4b1a-8657-d23c08c25edd None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:49.417786 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142eab21-a3a5-4b1a-8657-d23c08c25edd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:49.426985 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142eab21-a3a5-4b1a-8657-d23c08c25edd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:49.429217 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142eab21-a3a5-4b1a-8657-d23c08c25edd None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:49.439133 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466981ce-3930-4bcc-ba8b-a5158179a82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:49.439808 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466981ce-3930-4bcc-ba8b-a5158179a82a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:49.440094 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466981ce-3930-4bcc-ba8b-a5158179a82a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:49.440373 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466981ce-3930-4bcc-ba8b-a5158179a82a None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:49.441088 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466981ce-3930-4bcc-ba8b-a5158179a82a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:49.441395 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466981ce-3930-4bcc-ba8b-a5158179a82a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:49.442895 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466981ce-3930-4bcc-ba8b-a5158179a82a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:49.446172 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-142eab21-a3a5-4b1a-8657-d23c08c25edd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:49.446172 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:49.446172 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:49.446172 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:49.446172 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:49.446172 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:49.446172 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:49.446172 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:49.446172 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:49.446172 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:49.446172 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:49.446172 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:49.446172 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:18:49.446172 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:18:49.446172 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:18:49.447831 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:18:49.447831 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:49.447831 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:49.447831 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:49.447831 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:49.447831 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:49.447831 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:49.447831 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:49.447831 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:49.447831 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:49.447831 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:49.447831 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:49.447831 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:49.447831 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:49.447831 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:49.447831 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:49.447831 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:49.449496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:18:49.449496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:18:49.449496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:18:49.449496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:18:49.449496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:18:49.449496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:18:49.449496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:49.449496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:49.449496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 457/932] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:18:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:49.450457 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 476/933] 10.222.0.65 () {56 vars in 1126 bytes} [Thu May 19 21:18:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:49.456112 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142eab21-a3a5-4b1a-8657-d23c08c25edd None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:49.459572 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466981ce-3930-4bcc-ba8b-a5158179a82a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:49.540488 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:49.553753 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:49.581989 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466981ce-3930-4bcc-ba8b-a5158179a82a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:49.591601 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142eab21-a3a5-4b1a-8657-d23c08c25edd None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:49.622110 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b593cfa-56d3-4a92-8532-1263906ef22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:49.623249 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b593cfa-56d3-4a92-8532-1263906ef22e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:49.623791 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b593cfa-56d3-4a92-8532-1263906ef22e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:49.624424 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b593cfa-56d3-4a92-8532-1263906ef22e None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:49.625381 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b593cfa-56d3-4a92-8532-1263906ef22e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:49.625951 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b593cfa-56d3-4a92-8532-1263906ef22e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:49.627752 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b593cfa-56d3-4a92-8532-1263906ef22e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:49.635165 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019f1dd9-05e3-4f21-b5e2-00c432eaa671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:49.635506 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 477/934] 10.222.0.65 () {56 vars in 1126 bytes} [Thu May 19 21:18:49 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:49.635846 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f1dd9-05e3-4f21-b5e2-00c432eaa671 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:49.636122 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f1dd9-05e3-4f21-b5e2-00c432eaa671 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:49.636393 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f1dd9-05e3-4f21-b5e2-00c432eaa671 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:49.637189 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019f1dd9-05e3-4f21-b5e2-00c432eaa671 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:49.637497 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019f1dd9-05e3-4f21-b5e2-00c432eaa671 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:49.644311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b593cfa-56d3-4a92-8532-1263906ef22e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:49.644838 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019f1dd9-05e3-4f21-b5e2-00c432eaa671 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:49.662599 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 458/935] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 21:18:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:18:49.668762 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019f1dd9-05e3-4f21-b5e2-00c432eaa671 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:49.787550 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b593cfa-56d3-4a92-8532-1263906ef22e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:49.794736 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019f1dd9-05e3-4f21-b5e2-00c432eaa671 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:49.826896 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d6a068-e1ed-49a7-b2f2-bc2aabc41eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:49.827706 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d6a068-e1ed-49a7-b2f2-bc2aabc41eed None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:49.827995 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d6a068-e1ed-49a7-b2f2-bc2aabc41eed None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:49.828277 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d6a068-e1ed-49a7-b2f2-bc2aabc41eed None admin] PATH_INFO: `/v3/endpoints` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:49.829030 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d6a068-e1ed-49a7-b2f2-bc2aabc41eed None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:49.830702 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d6a068-e1ed-49a7-b2f2-bc2aabc41eed None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:49.836182 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:49.840918 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067cbdd8-37ce-4504-84d2-6fb3970d80c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:49.842179 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067cbdd8-37ce-4504-84d2-6fb3970d80c0 None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:49.842840 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067cbdd8-37ce-4504-84d2-6fb3970d80c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:49.843948 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067cbdd8-37ce-4504-84d2-6fb3970d80c0 None admin] PATH_INFO: `/v3/projects/c4940befa9c4405699c72deedc8a2127/users/eba91dff7cac401e81f6fb6391874af1/roles/bdd2ca99a0d74237b4642eae68a9a1c5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:49.850150 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067cbdd8-37ce-4504-84d2-6fb3970d80c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4940befa9c4405699c72deedc8a2127, user_id=eba91dff7cac401e81f6fb6391874af1, role_id=bdd2ca99a0d74237b4642eae68a9a1c5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:49.850150 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067cbdd8-37ce-4504-84d2-6fb3970d80c0 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:49.857374 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:49.870384 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 478/936] 10.222.0.65 () {60 vars in 1144 bytes} [Thu May 19 21:18:49 2022] POST /identity/v3/endpoints => generated 347 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:18:49.889927 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 459/937] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 21:18:49 2022] PUT /identity/v3/projects/c4940befa9c4405699c72deedc8a2127/users/eba91dff7cac401e81f6fb6391874af1/roles/bdd2ca99a0d74237b4642eae68a9a1c5 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:18:49.967800 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9b09a-9a26-48a6-b3e9-252033c48c27 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:49.968350 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9b09a-9a26-48a6-b3e9-252033c48c27 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:49.968660 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9b09a-9a26-48a6-b3e9-252033c48c27 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:49.971767 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 479/938] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:18:49.981439 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e87fa-a363-43d8-85e8-2de1cd895143 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:49.981927 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e87fa-a363-43d8-85e8-2de1cd895143 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:49.982469 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e87fa-a363-43d8-85e8-2de1cd895143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:50.015413 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-113e87fa-a363-43d8-85e8-2de1cd895143 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:50.017829 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-113e87fa-a363-43d8-85e8-2de1cd895143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:50.157643 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 460/939] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:18:50.194025 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e9b09a-9a26-48a6-b3e9-252033c48c27 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:50.383858 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e9b09a-9a26-48a6-b3e9-252033c48c27 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:50.426550 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd82ed-d638-479b-8fcb-cbe78aaeaea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:50.427457 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd82ed-d638-479b-8fcb-cbe78aaeaea8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:50.427737 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd82ed-d638-479b-8fcb-cbe78aaeaea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:50.428010 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd82ed-d638-479b-8fcb-cbe78aaeaea8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:50.428704 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd82ed-d638-479b-8fcb-cbe78aaeaea8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:50.430105 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd82ed-d638-479b-8fcb-cbe78aaeaea8 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:50.443087 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-82cd82ed-d638-479b-8fcb-cbe78aaeaea8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 21:18:50.443087 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:50.443087 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:18:50.443087 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:18:50.443087 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:18:50.443087 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:18:50.443087 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:18:50.443087 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:18:50.443087 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 21:18:50.443087 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:50.443087 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:18:50.443087 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:50.443087 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:50.443087 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:50.443087 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:50.443087 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:50.443087 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:50.444151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:50.444151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:50.444151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:50.444151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:50.444151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:50.444151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:50.444151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:18:50.444151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:18:50.444151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:18:50.444151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:18:50.444151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:50.444151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:50.444151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:50.444151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:50.444151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:50.444151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:50.445435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:50.445435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:50.445435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:50.445435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:50.445435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:50.445435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:50.445435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:50.445435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:50.445435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:50.445435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:50.445435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:18:50.445435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:18:50.445435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 21:18:50.445435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:50.445435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 480/940] 10.222.0.65 () {56 vars in 1126 bytes} [Thu May 19 21:18:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:50.450998 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e87fa-a363-43d8-85e8-2de1cd895143 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:50.568230 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e87fa-a363-43d8-85e8-2de1cd895143 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:50.593336 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:50.609670 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba84b57-94f7-49cb-9114-eefc84f6b64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:50.610388 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba84b57-94f7-49cb-9114-eefc84f6b64e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:50.610666 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba84b57-94f7-49cb-9114-eefc84f6b64e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:50.610941 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba84b57-94f7-49cb-9114-eefc84f6b64e None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:50.611663 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba84b57-94f7-49cb-9114-eefc84f6b64e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:50.611966 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba84b57-94f7-49cb-9114-eefc84f6b64e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:50.613439 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba84b57-94f7-49cb-9114-eefc84f6b64e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:50.620118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 461/941] 10.222.0.65 () {56 vars in 1122 bytes} [Thu May 19 21:18:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:50.625974 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd82ed-d638-479b-8fcb-cbe78aaeaea8 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:50.687983 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:50.740106 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd82ed-d638-479b-8fcb-cbe78aaeaea8 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:50.779219 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52915d8f-67ea-48de-831f-e5bfcb8998c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:50.779906 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52915d8f-67ea-48de-831f-e5bfcb8998c9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:50.780187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52915d8f-67ea-48de-831f-e5bfcb8998c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:50.780456 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52915d8f-67ea-48de-831f-e5bfcb8998c9 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:50.781176 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52915d8f-67ea-48de-831f-e5bfcb8998c9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:50.781480 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52915d8f-67ea-48de-831f-e5bfcb8998c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:50.782956 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52915d8f-67ea-48de-831f-e5bfcb8998c9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:50.789230 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 481/942] 10.222.0.65 () {56 vars in 1122 bytes} [Thu May 19 21:18:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:50.795163 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba84b57-94f7-49cb-9114-eefc84f6b64e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:50.908755 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba84b57-94f7-49cb-9114-eefc84f6b64e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:50.945891 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5bbca2-4de8-45c5-8e13-2dfca9f6d55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:50.946521 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5bbca2-4de8-45c5-8e13-2dfca9f6d55b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:50.946941 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5bbca2-4de8-45c5-8e13-2dfca9f6d55b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:50.947218 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5bbca2-4de8-45c5-8e13-2dfca9f6d55b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:50.948028 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5bbca2-4de8-45c5-8e13-2dfca9f6d55b None admin] RBAC: Adding query filter params (interface=public, service_id=88e7476037f94959a2974ed9dbf15390, region_id=RegionOne) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:50.948337 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5bbca2-4de8-45c5-8e13-2dfca9f6d55b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:50.949721 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5bbca2-4de8-45c5-8e13-2dfca9f6d55b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:50.956080 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 462/943] 10.222.0.65 () {56 vars in 1251 bytes} [Thu May 19 21:18:50 2022] GET /identity/v3/endpoints?service_id=88e7476037f94959a2974ed9dbf15390&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:50.961915 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52915d8f-67ea-48de-831f-e5bfcb8998c9 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:51.075319 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52915d8f-67ea-48de-831f-e5bfcb8998c9 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:51.098805 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:51.115091 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf96e647-3c89-4e8e-a846-5cbed900067c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:51.115780 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96e647-3c89-4e8e-a846-5cbed900067c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:51.116057 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96e647-3c89-4e8e-a846-5cbed900067c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:51.116320 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96e647-3c89-4e8e-a846-5cbed900067c None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:51.117014 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf96e647-3c89-4e8e-a846-5cbed900067c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:51.118425 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf96e647-3c89-4e8e-a846-5cbed900067c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:51.125555 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 482/944] 10.222.0.65 () {56 vars in 1087 bytes} [Thu May 19 21:18:50 2022] GET /identity/v3/services => generated 1497 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:18:51.838876 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba49666-ba7c-4d90-8506-4ebbaa4be102 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:51.839251 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba49666-ba7c-4d90-8506-4ebbaa4be102 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:51.839434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba49666-ba7c-4d90-8506-4ebbaa4be102 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:51.841166 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 463/945] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:18:51.847230 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d03981-9876-4c1f-9c18-0603d275412b None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:51.847536 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d03981-9876-4c1f-9c18-0603d275412b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:51.847786 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d03981-9876-4c1f-9c18-0603d275412b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:51.866805 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-00d03981-9876-4c1f-9c18-0603d275412b None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:51.868021 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-00d03981-9876-4c1f-9c18-0603d275412b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:51.948364 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 483/946] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:18:51.972605 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba49666-ba7c-4d90-8506-4ebbaa4be102 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:52.069428 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:52.185803 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba49666-ba7c-4d90-8506-4ebbaa4be102 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:52.249536 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657c6d9-8a85-4e4f-965d-4dd13729ab0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:52.250613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657c6d9-8a85-4e4f-965d-4dd13729ab0b None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:52.251116 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657c6d9-8a85-4e4f-965d-4dd13729ab0b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:52.251563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657c6d9-8a85-4e4f-965d-4dd13729ab0b None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:52.252471 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657c6d9-8a85-4e4f-965d-4dd13729ab0b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:52.254124 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657c6d9-8a85-4e4f-965d-4dd13729ab0b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:52.255833 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: WARNING keystone.server.flask.common [None req-5657c6d9-8a85-4e4f-965d-4dd13729ab0b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 21:18:52.277578 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.sql.core [None req-5657c6d9-8a85-4e4f-965d-4dd13729ab0b None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 21:18:52.277578 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 21:18:52.277578 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [parameters: {'id': 'c40d847d9fe148c28dddb8e1b6a09a95', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 21:18:52.277578 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80270) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 21:18:52.286442 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-5657c6d9-8a85-4e4f-965d-4dd13729ab0b None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c40d847d9fe148c28dddb8e1b6a09a95.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c40d847d9fe148c28dddb8e1b6a09a95. May 19 21:18:52.286442 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:52.286442 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 21:18:52.286442 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 21:18:52.286442 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 21:18:52.286442 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 21:18:52.286442 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 21:18:52.286442 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application result = self._query(query) May 19 21:18:52.286442 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 21:18:52.286442 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application conn.query(q) May 19 21:18:52.286442 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 21:18:52.286442 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 21:18:52.286442 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 21:18:52.287434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application result.read() May 19 21:18:52.287434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 21:18:52.287434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 21:18:52.287434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 21:18:52.287434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 21:18:52.287434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 21:18:52.287434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 21:18:52.287434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 21:18:52.287434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 21:18:52.287434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 21:18:52.287434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:52.287434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 21:18:52.287434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:52.287434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:52.287434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 21:18:52.287434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 21:18:52.287434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 19 21:18:52.287434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return project_ref.to_dict() May 19 21:18:52.288408 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 21:18:52.288408 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application next(self.gen) May 19 21:18:52.288408 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 21:18:52.288408 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application yield resource May 19 21:18:52.288408 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 21:18:52.288408 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application next(self.gen) May 19 21:18:52.288408 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 21:18:52.288408 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self.session.rollback() May 19 21:18:52.288408 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 21:18:52.288408 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self.force_reraise() May 19 21:18:52.288408 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 21:18:52.288408 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise self.value May 19 21:18:52.288408 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 21:18:52.288408 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 21:18:52.288408 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 21:18:52.288408 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application session.commit() May 19 21:18:52.288408 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 21:18:52.289315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 21:18:52.289315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 21:18:52.289315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._prepare_impl() May 19 21:18:52.289315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 21:18:52.289315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self.session.flush() May 19 21:18:52.289315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 21:18:52.289315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._flush(objects) May 19 21:18:52.289315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 21:18:52.289315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 21:18:52.289315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 21:18:52.289315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application compat.raise_( May 19 21:18:52.289315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 21:18:52.289315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception May 19 21:18:52.289315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 21:18:52.289315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application flush_context.execute() May 19 21:18:52.289315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 21:18:52.289315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rec.execute(self) May 19 21:18:52.290226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 21:18:52.290226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 21:18:52.290226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 21:18:52.290226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 21:18:52.290226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 21:18:52.290226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 21:18:52.290226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 21:18:52.290226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 21:18:52.290226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 21:18:52.290226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 21:18:52.290226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 21:18:52.290226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 21:18:52.290226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 21:18:52.290226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 21:18:52.290226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 21:18:52.291082 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 21:18:52.291082 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 21:18:52.291082 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception May 19 21:18:52.291082 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 21:18:52.291082 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 21:18:52.291082 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 21:18:52.291082 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 21:18:52.291082 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 21:18:52.291082 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application result = self._query(query) May 19 21:18:52.291082 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 21:18:52.291082 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application conn.query(q) May 19 21:18:52.291082 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 21:18:52.291082 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 21:18:52.291082 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 21:18:52.291082 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application result.read() May 19 21:18:52.291082 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 21:18:52.291082 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 21:18:52.291958 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 21:18:52.291958 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 21:18:52.291958 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 21:18:52.291958 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 21:18:52.291958 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 21:18:52.291958 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 21:18:52.291958 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 21:18:52.291958 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 21:18:52.291958 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [parameters: {'id': 'c40d847d9fe148c28dddb8e1b6a09a95', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 21:18:52.291958 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 21:18:52.291958 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:52.291958 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:18:52.291958 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:52.291958 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:52.291958 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 19 21:18:52.292734 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 19 21:18:52.292734 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 21:18:52.292734 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 21:18:52.292734 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 19 21:18:52.292734 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:52.292734 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:18:52.292734 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:52.292734 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:52.292734 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:52.292734 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:52.292734 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:52.292734 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:52.292734 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:52.292734 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:52.292734 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:52.292734 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:52.292734 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:52.293667 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:52.293667 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 19 21:18:52.293667 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 19 21:18:52.293667 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:52.293667 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:52.293667 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 19 21:18:52.293667 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.Conflict( May 19 21:18:52.293667 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c40d847d9fe148c28dddb8e1b6a09a95. May 19 21:18:52.293667 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:52.296561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 464/947] 10.222.0.65 () {60 vars in 1140 bytes} [Thu May 19 21:18:51 2022] POST /identity/v3/projects => generated 262 bytes in 325 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 19 21:18:52.304119 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d03981-9876-4c1f-9c18-0603d275412b None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:52.355608 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c080e-7705-41b4-9f67-29b893c4abbf None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:52.356520 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c080e-7705-41b4-9f67-29b893c4abbf None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:52.357576 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c080e-7705-41b4-9f67-29b893c4abbf None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:52.365456 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 465/948] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:52 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:18:52.374013 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7799e-7696-454a-9cf7-eaabbf6fae57 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:52.374424 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7799e-7696-454a-9cf7-eaabbf6fae57 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:52.374874 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7799e-7696-454a-9cf7-eaabbf6fae57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:52.400580 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-9cd7799e-7696-454a-9cf7-eaabbf6fae57 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:52.403279 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9cd7799e-7696-454a-9cf7-eaabbf6fae57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:52.461093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d03981-9876-4c1f-9c18-0603d275412b None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:52.487950 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:52.513097 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 466/949] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:18:52.535302 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6cfc90-a7d4-4d53-bff4-1458bcec9bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:52.536549 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6cfc90-a7d4-4d53-bff4-1458bcec9bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:52.536839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6cfc90-a7d4-4d53-bff4-1458bcec9bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:52.537208 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6cfc90-a7d4-4d53-bff4-1458bcec9bd2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:52.545512 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6cfc90-a7d4-4d53-bff4-1458bcec9bd2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:52.547046 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6cfc90-a7d4-4d53-bff4-1458bcec9bd2 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:52.551705 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd7799e-7696-454a-9cf7-eaabbf6fae57 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:52.563019 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-da6cfc90-a7d4-4d53-bff4-1458bcec9bd2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:52.563019 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:52.563019 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:52.563019 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:52.563019 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:52.563019 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:52.563019 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:52.563019 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:52.563019 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:52.563019 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:52.563019 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:52.563019 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:52.563019 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:18:52.563019 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:18:52.563019 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:18:52.564773 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:18:52.564773 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:52.564773 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:52.564773 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:52.564773 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:52.564773 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:52.564773 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:52.564773 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:52.564773 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:52.564773 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:52.564773 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:52.564773 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:52.564773 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:52.564773 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:52.564773 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:52.564773 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:52.564773 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:52.566663 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:18:52.566663 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:18:52.566663 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:18:52.566663 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:18:52.566663 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:18:52.566663 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:18:52.566663 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:52.566663 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:52.566663 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 484/950] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:18:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:18:52.571025 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6cfc90-a7d4-4d53-bff4-1458bcec9bd2 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:52.619692 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:52.689911 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6cfc90-a7d4-4d53-bff4-1458bcec9bd2 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:52.728911 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56990c80-059b-459f-8c82-12ca662af0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:52.729493 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56990c80-059b-459f-8c82-12ca662af0c5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:52.729770 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56990c80-059b-459f-8c82-12ca662af0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:52.730092 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56990c80-059b-459f-8c82-12ca662af0c5 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:52.730858 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56990c80-059b-459f-8c82-12ca662af0c5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:52.731167 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56990c80-059b-459f-8c82-12ca662af0c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:52.732660 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56990c80-059b-459f-8c82-12ca662af0c5 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:52.750512 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd7799e-7696-454a-9cf7-eaabbf6fae57 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:52.758420 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 485/951] 10.222.0.65 () {56 vars in 1112 bytes} [Thu May 19 21:18:52 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:18:52.827066 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b0097c-0fde-45b7-9d08-189a8314efc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:52.829006 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b0097c-0fde-45b7-9d08-189a8314efc0 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:52.829917 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b0097c-0fde-45b7-9d08-189a8314efc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:52.830745 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b0097c-0fde-45b7-9d08-189a8314efc0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:52.850763 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b0097c-0fde-45b7-9d08-189a8314efc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:18:52.866796 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-52b0097c-0fde-45b7-9d08-189a8314efc0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 21:18:52.866796 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:52.866796 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:52.866796 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:52.866796 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:52.866796 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:52.866796 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:52.866796 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:52.866796 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:52.866796 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:52.866796 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:52.866796 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:52.866796 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:18:52.866796 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:18:52.866796 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:18:52.866796 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:18:52.867884 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:18:52.867884 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:18:52.867884 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:18:52.867884 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:18:52.867884 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:52.867884 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:52.867884 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:52.867884 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:52.867884 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:52.867884 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:52.867884 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:52.867884 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:52.867884 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:52.867884 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:52.867884 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:52.867884 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:52.868769 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:52.868769 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:52.868769 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:52.868769 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:52.868769 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:18:52.868769 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:18:52.868769 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:18:52.868769 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:18:52.868769 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:18:52.868769 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:18:52.868769 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 21:18:52.868769 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:52.870036 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 467/952] 10.222.0.65 () {56 vars in 1120 bytes} [Thu May 19 21:18:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:52.878571 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56990c80-059b-459f-8c82-12ca662af0c5 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:52.934997 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:53.047038 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56990c80-059b-459f-8c82-12ca662af0c5 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:53.111977 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329aac16-f161-496f-9ffe-6f656b2091fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:53.113650 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329aac16-f161-496f-9ffe-6f656b2091fc None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:53.115865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329aac16-f161-496f-9ffe-6f656b2091fc None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:53.116669 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329aac16-f161-496f-9ffe-6f656b2091fc None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:53.118321 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329aac16-f161-496f-9ffe-6f656b2091fc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:53.119121 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329aac16-f161-496f-9ffe-6f656b2091fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:53.120823 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329aac16-f161-496f-9ffe-6f656b2091fc None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:53.134383 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 486/953] 10.222.0.65 () {56 vars in 1115 bytes} [Thu May 19 21:18:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:53.144234 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b0097c-0fde-45b7-9d08-189a8314efc0 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:53.170594 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5136b9c8-0f86-49d1-ad81-ec96fb90db59 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:53.171230 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5136b9c8-0f86-49d1-ad81-ec96fb90db59 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:53.171729 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5136b9c8-0f86-49d1-ad81-ec96fb90db59 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:53.173716 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 487/954] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:18:53.179282 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab20dd1-697d-4cd0-a8f2-fa1ed08a5388 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:53.179587 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab20dd1-697d-4cd0-a8f2-fa1ed08a5388 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:53.179839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab20dd1-697d-4cd0-a8f2-fa1ed08a5388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:53.198757 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-eab20dd1-697d-4cd0-a8f2-fa1ed08a5388 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:53.199967 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-eab20dd1-697d-4cd0-a8f2-fa1ed08a5388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:53.317085 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 488/955] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:18:53.334412 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab20dd1-697d-4cd0-a8f2-fa1ed08a5388 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:53.346952 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:53.360172 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b0097c-0fde-45b7-9d08-189a8314efc0 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:53.372603 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:53.407163 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c76662-400b-4946-be68-f29e4acc208c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:53.408292 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c76662-400b-4946-be68-f29e4acc208c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:53.408741 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c76662-400b-4946-be68-f29e4acc208c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:53.409179 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c76662-400b-4946-be68-f29e4acc208c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:53.421816 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c76662-400b-4946-be68-f29e4acc208c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:53.423719 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c76662-400b-4946-be68-f29e4acc208c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:53.455298 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-c9c76662-400b-4946-be68-f29e4acc208c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:53.455298 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:53.455298 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:53.455298 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:53.455298 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:53.455298 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:53.455298 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:53.455298 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:53.455298 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:53.455298 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:53.455298 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:53.455298 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:53.455298 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:53.455298 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:53.455298 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:53.456856 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:53.456856 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:53.456856 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:53.456856 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:53.456856 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:53.456856 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:53.456856 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:53.456856 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:53.456856 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:53.456856 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:53.456856 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:53.456856 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:53.456856 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:53.456856 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:53.456856 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:53.456856 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:53.456856 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:53.459016 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:53.459016 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:53.459016 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:53.459016 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:53.462380 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 468/956] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:53.475511 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c76662-400b-4946-be68-f29e4acc208c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:53.510893 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab20dd1-697d-4cd0-a8f2-fa1ed08a5388 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:53.565462 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6a7eb-7444-4777-aeae-df9317b2575c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:53.566863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6a7eb-7444-4777-aeae-df9317b2575c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:53.567439 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6a7eb-7444-4777-aeae-df9317b2575c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:53.567891 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6a7eb-7444-4777-aeae-df9317b2575c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:53.569676 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6a7eb-7444-4777-aeae-df9317b2575c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:53.571323 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6a7eb-7444-4777-aeae-df9317b2575c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:53.604066 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-91c6a7eb-7444-4777-aeae-df9317b2575c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 21:18:53.604066 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:53.604066 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:18:53.604066 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:18:53.604066 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:18:53.604066 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:18:53.604066 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:18:53.604066 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:18:53.604066 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 21:18:53.604066 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:53.604066 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:18:53.604066 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:53.604066 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:53.604066 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:53.604066 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:53.604066 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:53.604066 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:53.606256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:53.606256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:53.606256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:53.606256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:53.606256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:53.606256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:53.606256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:18:53.606256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:18:53.606256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:18:53.606256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:18:53.606256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:53.606256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:53.606256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:53.606256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:53.606256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:53.606256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:53.607950 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:53.607950 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:53.607950 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:53.607950 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:53.607950 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:53.607950 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:53.607950 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:53.607950 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:53.607950 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:53.607950 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:53.607950 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:18:53.607950 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:18:53.607950 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 21:18:53.607950 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:53.609341 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 489/957] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:18:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:18:53.621341 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6a7eb-7444-4777-aeae-df9317b2575c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:53.661688 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c76662-400b-4946-be68-f29e4acc208c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:53.728721 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:53.731452 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e29175-881a-49e4-ad8f-dd8ed3ed1f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:53.733003 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e29175-881a-49e4-ad8f-dd8ed3ed1f4b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:53.733836 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e29175-881a-49e4-ad8f-dd8ed3ed1f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:53.734999 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e29175-881a-49e4-ad8f-dd8ed3ed1f4b None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:53.737447 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e29175-881a-49e4-ad8f-dd8ed3ed1f4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:53.738429 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e29175-881a-49e4-ad8f-dd8ed3ed1f4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:53.739859 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6a7eb-7444-4777-aeae-df9317b2575c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:53.742555 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e29175-881a-49e4-ad8f-dd8ed3ed1f4b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:53.769470 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 469/958] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:53.776053 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e29175-881a-49e4-ad8f-dd8ed3ed1f4b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:53.784227 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ab5489-916e-4580-bff6-d7324a64e77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:53.784933 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab5489-916e-4580-bff6-d7324a64e77f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:53.785233 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab5489-916e-4580-bff6-d7324a64e77f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:53.785478 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab5489-916e-4580-bff6-d7324a64e77f None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:53.786452 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ab5489-916e-4580-bff6-d7324a64e77f None admin] RBAC: Adding query filter params (name=compute) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:53.786767 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ab5489-916e-4580-bff6-d7324a64e77f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:53.788386 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ab5489-916e-4580-bff6-d7324a64e77f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:53.794159 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:53.799494 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 490/959] 10.222.0.65 () {56 vars in 1112 bytes} [Thu May 19 21:18:53 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:18:53.800779 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ab5489-916e-4580-bff6-d7324a64e77f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:53.917952 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ab5489-916e-4580-bff6-d7324a64e77f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:53.955765 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95979065-d819-47c8-a7b7-fb0fceaae891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:53.956465 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95979065-d819-47c8-a7b7-fb0fceaae891 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:53.956752 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95979065-d819-47c8-a7b7-fb0fceaae891 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:53.957024 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95979065-d819-47c8-a7b7-fb0fceaae891 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:53.957749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95979065-d819-47c8-a7b7-fb0fceaae891 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:53.958127 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95979065-d819-47c8-a7b7-fb0fceaae891 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:53.959960 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95979065-d819-47c8-a7b7-fb0fceaae891 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:53.970724 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 491/960] 10.222.0.65 () {56 vars in 1112 bytes} [Thu May 19 21:18:53 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:53.985841 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e29175-881a-49e4-ad8f-dd8ed3ed1f4b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:54.054350 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd529788-35bb-4679-ae8c-5886a91a22c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:54.055687 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd529788-35bb-4679-ae8c-5886a91a22c9 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:54.056192 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd529788-35bb-4679-ae8c-5886a91a22c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:54.056691 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd529788-35bb-4679-ae8c-5886a91a22c9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:54.064232 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1a7c5-bda9-400b-921e-08ce7b91a4b0 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:54.064635 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1a7c5-bda9-400b-921e-08ce7b91a4b0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:54.064790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1a7c5-bda9-400b-921e-08ce7b91a4b0 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:54.066202 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd529788-35bb-4679-ae8c-5886a91a22c9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:54.067821 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd529788-35bb-4679-ae8c-5886a91a22c9 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:54.070520 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 492/961] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:18:54.077464 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5632b5-3e2d-4201-b07d-4586fca83517 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:54.078964 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5632b5-3e2d-4201-b07d-4586fca83517 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:54.079769 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5632b5-3e2d-4201-b07d-4586fca83517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:54.088300 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-bd529788-35bb-4679-ae8c-5886a91a22c9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 21:18:54.088300 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:54.088300 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:54.088300 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:54.088300 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:54.088300 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:54.088300 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:54.088300 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:54.088300 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:54.088300 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:54.088300 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:54.088300 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:54.088300 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:18:54.088300 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:18:54.088300 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:18:54.088300 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:18:54.090195 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:54.090195 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:54.090195 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:18:54.090195 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:54.090195 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:18:54.090195 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:54.090195 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:54.090195 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:54.090195 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:54.090195 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:54.090195 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:54.090195 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:54.090195 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:54.090195 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:54.090195 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:54.090195 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:54.090195 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:54.090195 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:54.092059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:54.092059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:54.092059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:18:54.092059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:18:54.092059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:18:54.092059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:18:54.092059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:18:54.092059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:18:54.092059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 21:18:54.092059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:54.092059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 470/962] 10.222.0.65 () {56 vars in 1117 bytes} [Thu May 19 21:18:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:18:54.096856 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd529788-35bb-4679-ae8c-5886a91a22c9 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:54.107537 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-7c5632b5-3e2d-4201-b07d-4586fca83517 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:54.107722 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7c5632b5-3e2d-4201-b07d-4586fca83517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:54.130219 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:54.234823 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:54.237356 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 493/963] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:18:54.260354 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5632b5-3e2d-4201-b07d-4586fca83517 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:54.294992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd529788-35bb-4679-ae8c-5886a91a22c9 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:54.351041 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be70108-a35c-4686-b919-9b6f90e39a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:54.352129 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be70108-a35c-4686-b919-9b6f90e39a59 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:54.352414 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be70108-a35c-4686-b919-9b6f90e39a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:54.353005 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be70108-a35c-4686-b919-9b6f90e39a59 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:54.353864 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be70108-a35c-4686-b919-9b6f90e39a59 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:54.354180 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be70108-a35c-4686-b919-9b6f90e39a59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:54.356394 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be70108-a35c-4686-b919-9b6f90e39a59 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:54.382110 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 471/964] 10.222.0.65 () {56 vars in 1149 bytes} [Thu May 19 21:18:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:18:54.387795 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be70108-a35c-4686-b919-9b6f90e39a59 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:54.408021 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5632b5-3e2d-4201-b07d-4586fca83517 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:54.450325 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11079aa2-319e-4469-966a-4919f2d1a062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:54.451160 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11079aa2-319e-4469-966a-4919f2d1a062 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:54.451439 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11079aa2-319e-4469-966a-4919f2d1a062 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:54.451714 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11079aa2-319e-4469-966a-4919f2d1a062 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:54.452380 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11079aa2-319e-4469-966a-4919f2d1a062 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:54.453770 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11079aa2-319e-4469-966a-4919f2d1a062 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:54.468296 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-11079aa2-319e-4469-966a-4919f2d1a062 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 21:18:54.468296 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:54.468296 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:18:54.468296 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:18:54.468296 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:18:54.468296 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:18:54.468296 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:18:54.468296 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:18:54.468296 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 21:18:54.468296 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:54.468296 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:18:54.468296 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:54.468296 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:54.468296 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:54.468296 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:54.468296 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:54.468296 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:54.469431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:54.469431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:54.469431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:54.469431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:54.469431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:54.469431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:54.469431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:18:54.469431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:18:54.469431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:18:54.469431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:18:54.469431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:54.469431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:54.469431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:54.469431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:54.469431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:54.469431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:54.470311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:54.470311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:54.470311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:54.470311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:54.470311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:54.470311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:54.470311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:54.470311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:54.470311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:54.470311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:54.470311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:18:54.470311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:18:54.470311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 21:18:54.470311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:54.470311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 494/965] 10.222.0.65 () {56 vars in 1126 bytes} [Thu May 19 21:18:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:18:54.479585 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11079aa2-319e-4469-966a-4919f2d1a062 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:54.503671 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:54.550119 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be70108-a35c-4686-b919-9b6f90e39a59 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:54.579895 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:54.597464 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12cd45b-e718-4089-8d14-7af4d67357bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:54.598317 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12cd45b-e718-4089-8d14-7af4d67357bc None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:54.598675 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12cd45b-e718-4089-8d14-7af4d67357bc None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:54.599155 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12cd45b-e718-4089-8d14-7af4d67357bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:54.637031 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11079aa2-319e-4469-966a-4919f2d1a062 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:54.693716 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a9e7c4-0d36-48c3-98a8-2554c992418e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:54.694997 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9e7c4-0d36-48c3-98a8-2554c992418e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:54.695481 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9e7c4-0d36-48c3-98a8-2554c992418e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:54.695771 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9e7c4-0d36-48c3-98a8-2554c992418e None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:54.697135 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a9e7c4-0d36-48c3-98a8-2554c992418e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:54.697635 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a9e7c4-0d36-48c3-98a8-2554c992418e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:54.700312 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a9e7c4-0d36-48c3-98a8-2554c992418e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:54.701556 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12cd45b-e718-4089-8d14-7af4d67357bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:54.704008 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12cd45b-e718-4089-8d14-7af4d67357bc None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:54.714948 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 495/966] 10.222.0.65 () {56 vars in 1122 bytes} [Thu May 19 21:18:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:54.718752 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a9e7c4-0d36-48c3-98a8-2554c992418e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:54.839732 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:54.865912 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 472/967] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:18:54 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 21:18:54.877182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12cd45b-e718-4089-8d14-7af4d67357bc None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:54.904723 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a9e7c4-0d36-48c3-98a8-2554c992418e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:54.917752 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:54.949784 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a108bac-bdec-4c34-af6c-acb3f67ac8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:54.950538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a108bac-bdec-4c34-af6c-acb3f67ac8e3 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:54.950821 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a108bac-bdec-4c34-af6c-acb3f67ac8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:54.951093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a108bac-bdec-4c34-af6c-acb3f67ac8e3 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:54.951859 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a108bac-bdec-4c34-af6c-acb3f67ac8e3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:54.952166 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a108bac-bdec-4c34-af6c-acb3f67ac8e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:54.953581 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a108bac-bdec-4c34-af6c-acb3f67ac8e3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:54.960363 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 496/968] 10.222.0.65 () {56 vars in 1122 bytes} [Thu May 19 21:18:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:54.966995 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a108bac-bdec-4c34-af6c-acb3f67ac8e3 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:55.030223 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12cd45b-e718-4089-8d14-7af4d67357bc None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:55.075901 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c333456-beb5-47f4-8413-6342b00da86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:55.076734 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c333456-beb5-47f4-8413-6342b00da86d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:55.077022 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c333456-beb5-47f4-8413-6342b00da86d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:55.077300 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c333456-beb5-47f4-8413-6342b00da86d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:55.090194 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c333456-beb5-47f4-8413-6342b00da86d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:55.091738 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c333456-beb5-47f4-8413-6342b00da86d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:55.115751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-4c333456-beb5-47f4-8413-6342b00da86d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:55.115751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:55.115751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:55.115751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:55.115751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:55.115751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:55.115751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:55.115751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:55.115751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:55.115751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:55.115751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:55.115751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:55.115751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:18:55.115751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:18:55.115751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:18:55.118069 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:18:55.118069 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:55.118069 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:55.118069 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:55.118069 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:55.118069 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:55.118069 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:55.118069 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:55.118069 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:55.118069 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:55.118069 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:55.118069 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:55.118069 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:55.118069 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:55.118069 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:55.118069 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:55.118069 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:55.120010 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:18:55.120010 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:18:55.120010 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:18:55.120010 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:55.120010 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 473/969] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:55.123280 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c333456-beb5-47f4-8413-6342b00da86d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:55.184478 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a108bac-bdec-4c34-af6c-acb3f67ac8e3 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:55.238813 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647c414-6f8f-42d2-9e11-bf9b29c3f391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:55.240391 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647c414-6f8f-42d2-9e11-bf9b29c3f391 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:55.240895 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647c414-6f8f-42d2-9e11-bf9b29c3f391 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:55.241390 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647c414-6f8f-42d2-9e11-bf9b29c3f391 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:55.242844 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c647c414-6f8f-42d2-9e11-bf9b29c3f391 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:55.245608 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c647c414-6f8f-42d2-9e11-bf9b29c3f391 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:55.263630 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:55.293276 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 497/970] 10.222.0.65 () {60 vars in 1144 bytes} [Thu May 19 21:18:54 2022] POST /identity/v3/endpoints => generated 349 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:18:55.318318 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:55.360121 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf7e27-bedc-4e12-9de9-17c378b85dd8 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:55.361304 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf7e27-bedc-4e12-9de9-17c378b85dd8 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:55.362233 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf7e27-bedc-4e12-9de9-17c378b85dd8 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:55.366550 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 498/971] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:55 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:18:55.367250 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c333456-beb5-47f4-8413-6342b00da86d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:55.373147 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8676a-67a6-40fe-8800-0c84a9cf2bd0 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:55.374265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8676a-67a6-40fe-8800-0c84a9cf2bd0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:55.375090 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8676a-67a6-40fe-8800-0c84a9cf2bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:55.408289 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-94c8676a-67a6-40fe-8800-0c84a9cf2bd0 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:55.412605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-94c8676a-67a6-40fe-8800-0c84a9cf2bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:55.450592 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e617e-fb8b-423e-be79-a7e050f3259b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:55.451612 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e617e-fb8b-423e-be79-a7e050f3259b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:55.452084 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e617e-fb8b-423e-be79-a7e050f3259b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:55.452565 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e617e-fb8b-423e-be79-a7e050f3259b None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:55.454167 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3e617e-fb8b-423e-be79-a7e050f3259b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:55.454745 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3e617e-fb8b-423e-be79-a7e050f3259b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:55.457504 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3e617e-fb8b-423e-be79-a7e050f3259b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:55.482011 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 474/972] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:18:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:55.483068 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e617e-fb8b-423e-be79-a7e050f3259b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:55.561929 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 499/973] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:18:55.592457 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8676a-67a6-40fe-8800-0c84a9cf2bd0 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:55.637679 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e617e-fb8b-423e-be79-a7e050f3259b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:55.696670 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:55.709264 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04db41-3d3f-4c71-adb5-90e6c4c956e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:55.711165 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04db41-3d3f-4c71-adb5-90e6c4c956e0 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:55.711977 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04db41-3d3f-4c71-adb5-90e6c4c956e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:55.712801 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04db41-3d3f-4c71-adb5-90e6c4c956e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:55.726278 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8676a-67a6-40fe-8800-0c84a9cf2bd0 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:55.729949 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a04db41-3d3f-4c71-adb5-90e6c4c956e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:55.732726 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a04db41-3d3f-4c71-adb5-90e6c4c956e0 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:55.750893 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:55.759337 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c654c2-c906-486e-86e5-7f8c3790b5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:55.760066 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c654c2-c906-486e-86e5-7f8c3790b5a9 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:55.760358 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c654c2-c906-486e-86e5-7f8c3790b5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:55.760607 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c654c2-c906-486e-86e5-7f8c3790b5a9 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:55.761449 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c654c2-c906-486e-86e5-7f8c3790b5a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:55.762991 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c654c2-c906-486e-86e5-7f8c3790b5a9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:55.763183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-4a04db41-3d3f-4c71-adb5-90e6c4c956e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:55.763183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:55.763183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:55.763183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:55.763183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:55.763183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:55.763183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:55.763183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:55.763183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:55.763183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:55.763183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:55.763183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:55.763183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:18:55.763183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:18:55.763183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:18:55.764800 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:18:55.764800 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:55.764800 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:55.764800 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:55.764800 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:55.764800 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:55.764800 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:55.764800 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:55.764800 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:55.764800 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:55.764800 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:55.764800 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:55.764800 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:55.764800 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:55.764800 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:55.764800 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:55.764800 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:55.768289 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:18:55.768289 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:18:55.768289 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:18:55.768289 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:18:55.768289 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:18:55.768289 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:18:55.768289 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:18:55.768289 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:55.769391 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: WARNING keystone.server.flask.common [None req-04c654c2-c906-486e-86e5-7f8c3790b5a9 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 21:18:55.774005 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 475/974] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:18:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:55.776106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04db41-3d3f-4c71-adb5-90e6c4c956e0 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:55.780843 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.sql.core [None req-04c654c2-c906-486e-86e5-7f8c3790b5a9 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 21:18:55.780843 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 21:18:55.780843 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [parameters: {'user_id': 'ebbf7984bbff427f9b6afa168bd9d81d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 21:18:55.780843 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80269) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 21:18:55.797839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-04c654c2-c906-486e-86e5-7f8c3790b5a9 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 21:18:55.797839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:55.797839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 21:18:55.797839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 21:18:55.797839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 21:18:55.797839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 21:18:55.797839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 21:18:55.797839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application result = self._query(query) May 19 21:18:55.797839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 21:18:55.797839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application conn.query(q) May 19 21:18:55.797839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 21:18:55.797839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 21:18:55.797839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 21:18:55.797839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application result.read() May 19 21:18:55.797839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 21:18:55.800167 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 21:18:55.800167 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 21:18:55.800167 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 21:18:55.800167 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 21:18:55.800167 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 21:18:55.800167 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 21:18:55.800167 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 21:18:55.800167 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 21:18:55.800167 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:55.800167 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 21:18:55.800167 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:55.800167 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:55.800167 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 21:18:55.800167 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 21:18:55.800167 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 19 21:18:55.800167 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 19 21:18:55.800167 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 21:18:55.800167 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application next(self.gen) May 19 21:18:55.802102 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 21:18:55.802102 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application yield resource May 19 21:18:55.802102 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 21:18:55.802102 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application next(self.gen) May 19 21:18:55.802102 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 21:18:55.802102 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self.session.rollback() May 19 21:18:55.802102 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 21:18:55.802102 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self.force_reraise() May 19 21:18:55.802102 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 21:18:55.802102 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise self.value May 19 21:18:55.802102 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 21:18:55.802102 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 21:18:55.802102 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 21:18:55.802102 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application session.commit() May 19 21:18:55.802102 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 21:18:55.802102 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 21:18:55.802102 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 21:18:55.807071 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._prepare_impl() May 19 21:18:55.807071 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 21:18:55.807071 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self.session.flush() May 19 21:18:55.807071 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 21:18:55.807071 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._flush(objects) May 19 21:18:55.807071 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 21:18:55.807071 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 21:18:55.807071 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 21:18:55.807071 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application compat.raise_( May 19 21:18:55.807071 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 21:18:55.807071 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception May 19 21:18:55.807071 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 21:18:55.807071 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application flush_context.execute() May 19 21:18:55.807071 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 21:18:55.807071 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rec.execute(self) May 19 21:18:55.807071 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 21:18:55.807071 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 21:18:55.814309 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 21:18:55.814309 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 21:18:55.814309 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 19 21:18:55.814309 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application result = connection._execute_20( May 19 21:18:55.814309 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 21:18:55.814309 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 21:18:55.814309 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 21:18:55.814309 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 21:18:55.814309 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 21:18:55.814309 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 21:18:55.814309 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 21:18:55.814309 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 21:18:55.814309 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 21:18:55.814309 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 21:18:55.814309 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 21:18:55.814309 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception May 19 21:18:55.824154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 21:18:55.824154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 21:18:55.824154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 21:18:55.824154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 21:18:55.824154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 21:18:55.824154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application result = self._query(query) May 19 21:18:55.824154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 21:18:55.824154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application conn.query(q) May 19 21:18:55.824154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 21:18:55.824154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 21:18:55.824154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 21:18:55.824154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application result.read() May 19 21:18:55.824154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 21:18:55.824154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 21:18:55.824154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 21:18:55.824154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 21:18:55.827664 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 21:18:55.827664 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 21:18:55.827664 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 21:18:55.827664 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 21:18:55.827664 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 21:18:55.827664 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 21:18:55.827664 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ebbf7984bbff427f9b6afa168bd9d81d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 21:18:55.827664 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 21:18:55.827664 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:55.827664 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:18:55.827664 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:55.827664 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:55.827664 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:55.827664 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:55.827664 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:55.831197 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:55.831197 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:55.831197 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:55.831197 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:55.831197 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:55.831197 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:55.831197 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:55.831197 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 19 21:18:55.831197 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 19 21:18:55.831197 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:55.831197 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:55.831197 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:18:55.831197 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:55.831197 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:18:55.831197 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:55.831197 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 19 21:18:55.831197 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 19 21:18:55.835043 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 19 21:18:55.835043 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 19 21:18:55.835043 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 21:18:55.835043 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 21:18:55.835043 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 21:18:55.835043 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:55.835043 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 500/975] 10.222.0.65 () {60 vars in 1131 bytes} [Thu May 19 21:18:55 2022] POST /identity/v3/users => generated 160 bytes in 208 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 19 21:18:55.835043 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c654c2-c906-486e-86e5-7f8c3790b5a9 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:55.921780 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04db41-3d3f-4c71-adb5-90e6c4c956e0 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:55.962737 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c775ef3a-41d7-42da-957e-fcefb3f4b204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:55.963334 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c775ef3a-41d7-42da-957e-fcefb3f4b204 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:55.963617 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c775ef3a-41d7-42da-957e-fcefb3f4b204 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:55.963869 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c775ef3a-41d7-42da-957e-fcefb3f4b204 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:55.964670 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c775ef3a-41d7-42da-957e-fcefb3f4b204 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:55.964978 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c775ef3a-41d7-42da-957e-fcefb3f4b204 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:55.966502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c775ef3a-41d7-42da-957e-fcefb3f4b204 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:55.980884 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 476/976] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 21:18:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:55.990859 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c775ef3a-41d7-42da-957e-fcefb3f4b204 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:56.064012 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c654c2-c906-486e-86e5-7f8c3790b5a9 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:56.090751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:56.132900 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a187cdf-e09a-4687-82a3-0bc491daba70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:56.133748 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a187cdf-e09a-4687-82a3-0bc491daba70 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:56.134041 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a187cdf-e09a-4687-82a3-0bc491daba70 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:56.134348 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a187cdf-e09a-4687-82a3-0bc491daba70 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:56.143666 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a187cdf-e09a-4687-82a3-0bc491daba70 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:56.146062 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a187cdf-e09a-4687-82a3-0bc491daba70 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:56.164056 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-9a187cdf-e09a-4687-82a3-0bc491daba70 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 21:18:56.164056 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:56.164056 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:56.164056 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:56.164056 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:56.164056 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:56.164056 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:56.164056 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:56.164056 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:56.164056 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:56.164056 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:56.164056 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:56.164056 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:18:56.164056 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:18:56.164056 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:18:56.164056 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:18:56.171993 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:56.171993 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:56.171993 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:18:56.171993 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:56.171993 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:18:56.171993 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:18:56.171993 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:56.171993 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:56.171993 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:56.171993 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:56.171993 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:56.171993 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:56.171993 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:56.171993 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:56.171993 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:56.171993 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:56.171993 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:56.171993 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:56.173857 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:56.173857 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:56.173857 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:18:56.173857 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:18:56.173857 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:18:56.173857 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:18:56.173857 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:18:56.173857 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:18:56.173857 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 21:18:56.173857 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:56.173857 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 501/977] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:18:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:18:56.181898 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a187cdf-e09a-4687-82a3-0bc491daba70 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:56.185580 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c775ef3a-41d7-42da-957e-fcefb3f4b204 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:56.227463 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12094cb-94b6-4b0f-8d8d-9d3e65a8b30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:56.228048 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12094cb-94b6-4b0f-8d8d-9d3e65a8b30e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:56.228315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12094cb-94b6-4b0f-8d8d-9d3e65a8b30e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:56.228595 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12094cb-94b6-4b0f-8d8d-9d3e65a8b30e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:56.229985 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12094cb-94b6-4b0f-8d8d-9d3e65a8b30e None admin] RBAC: Adding query filter params (role.id=bdd2ca99a0d74237b4642eae68a9a1c5, scope.project.id=c4940befa9c4405699c72deedc8a2127, user.id=eba91dff7cac401e81f6fb6391874af1) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:56.229985 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12094cb-94b6-4b0f-8d8d-9d3e65a8b30e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:56.231330 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12094cb-94b6-4b0f-8d8d-9d3e65a8b30e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:56.238808 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 477/978] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:18:55 2022] GET /identity/v3/role_assignments?user.id=eba91dff7cac401e81f6fb6391874af1&scope.project.id=c4940befa9c4405699c72deedc8a2127&role.id=bdd2ca99a0d74237b4642eae68a9a1c5 => generated 608 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:56.295996 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:56.405448 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b28c2-8e41-425c-838c-1fb7c9fd1d13 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:56.406031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b28c2-8e41-425c-838c-1fb7c9fd1d13 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:56.406031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b28c2-8e41-425c-838c-1fb7c9fd1d13 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:56.407821 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 478/979] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:18:56.412569 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a187cdf-e09a-4687-82a3-0bc491daba70 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:56.414793 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eece35-8a4e-4c23-8916-80066fcf91bc None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:56.415381 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eece35-8a4e-4c23-8916-80066fcf91bc None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:56.415900 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eece35-8a4e-4c23-8916-80066fcf91bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:56.435664 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b6eece35-8a4e-4c23-8916-80066fcf91bc None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:56.437251 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b6eece35-8a4e-4c23-8916-80066fcf91bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:56.454861 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041125f5-e2d4-4c13-aec3-ae9e5f5676b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:56.455997 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041125f5-e2d4-4c13-aec3-ae9e5f5676b1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:56.456288 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041125f5-e2d4-4c13-aec3-ae9e5f5676b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:56.456570 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041125f5-e2d4-4c13-aec3-ae9e5f5676b1 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:56.458279 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041125f5-e2d4-4c13-aec3-ae9e5f5676b1 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:56.458279 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041125f5-e2d4-4c13-aec3-ae9e5f5676b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:56.459511 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041125f5-e2d4-4c13-aec3-ae9e5f5676b1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:56.501937 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 502/980] 10.222.0.65 () {56 vars in 1101 bytes} [Thu May 19 21:18:56 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:18:56.526805 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 479/981] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:18:56.550538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041125f5-e2d4-4c13-aec3-ae9e5f5676b1 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:56.698497 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041125f5-e2d4-4c13-aec3-ae9e5f5676b1 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:56.707696 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:56.739644 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c105e-cd87-4855-a793-e2fb3f09a7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:56.740598 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c105e-cd87-4855-a793-e2fb3f09a7ed None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:56.741048 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c105e-cd87-4855-a793-e2fb3f09a7ed None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:56.741477 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c105e-cd87-4855-a793-e2fb3f09a7ed None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:56.742445 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54c105e-cd87-4855-a793-e2fb3f09a7ed None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:56.744027 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54c105e-cd87-4855-a793-e2fb3f09a7ed None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:56.761188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 503/982] 10.222.0.65 () {60 vars in 1141 bytes} [Thu May 19 21:18:56 2022] POST /identity/v3/services => generated 242 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:18:58.437315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a4a12-934e-4528-a421-492d248e900e None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:58.438186 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a4a12-934e-4528-a421-492d248e900e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:58.438186 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a4a12-934e-4528-a421-492d248e900e None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:58.439692 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 480/983] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:18:58.445878 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346cda1-6e23-47b9-9884-0f2433554811 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:58.446725 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346cda1-6e23-47b9-9884-0f2433554811 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:58.446725 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346cda1-6e23-47b9-9884-0f2433554811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:58.464865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b346cda1-6e23-47b9-9884-0f2433554811 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:58.466410 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b346cda1-6e23-47b9-9884-0f2433554811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:58.554664 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 504/984] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:58 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:18:58.578383 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a4a12-934e-4528-a421-492d248e900e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:58.640342 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:58.704748 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a4a12-934e-4528-a421-492d248e900e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:58.747494 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f2199-fda1-4af0-9c66-334df8d66b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:58.748319 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f2199-fda1-4af0-9c66-334df8d66b29 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:58.748799 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f2199-fda1-4af0-9c66-334df8d66b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:58.748967 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f2199-fda1-4af0-9c66-334df8d66b29 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:58.749500 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f2199-fda1-4af0-9c66-334df8d66b29 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:58.750913 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f2199-fda1-4af0-9c66-334df8d66b29 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:58.765323 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-8d8f2199-fda1-4af0-9c66-334df8d66b29 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 21:18:58.765323 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:58.765323 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:18:58.765323 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:18:58.765323 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:18:58.765323 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:18:58.765323 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:18:58.765323 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:18:58.765323 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 21:18:58.765323 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:58.765323 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:18:58.765323 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:58.765323 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:58.765323 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:58.765323 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:58.765323 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:58.765323 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:58.775526 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:58.775526 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:58.775526 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:58.775526 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:58.775526 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:58.775526 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:58.775526 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:18:58.775526 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:18:58.775526 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:18:58.775526 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:18:58.775526 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:58.775526 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:58.775526 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:58.775526 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:58.775526 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:58.775526 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:58.777679 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:58.777679 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:18:58.777679 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:58.777679 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:58.777679 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:58.777679 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:58.777679 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:58.777679 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:58.777679 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:58.777679 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:58.777679 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:18:58.777679 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:18:58.777679 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 21:18:58.777679 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:18:58.777679 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 481/985] 10.222.0.65 () {56 vars in 1126 bytes} [Thu May 19 21:18:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:18:58.782494 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b346cda1-6e23-47b9-9884-0f2433554811 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:58.965235 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:58.986902 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b346cda1-6e23-47b9-9884-0f2433554811 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:59.058363 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d367ce18-0563-41b6-b587-63c1e6909013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:59.059529 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367ce18-0563-41b6-b587-63c1e6909013 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:59.060035 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367ce18-0563-41b6-b587-63c1e6909013 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:59.060534 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367ce18-0563-41b6-b587-63c1e6909013 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:59.061856 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d367ce18-0563-41b6-b587-63c1e6909013 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:59.062431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d367ce18-0563-41b6-b587-63c1e6909013 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:59.065008 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d367ce18-0563-41b6-b587-63c1e6909013 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:59.082289 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 505/986] 10.222.0.65 () {56 vars in 1122 bytes} [Thu May 19 21:18:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:59.086655 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f2199-fda1-4af0-9c66-334df8d66b29 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:59.154954 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065965e-5c66-4abc-b016-e8b4da99c5a2 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:59.155276 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065965e-5c66-4abc-b016-e8b4da99c5a2 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:59.155685 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065965e-5c66-4abc-b016-e8b4da99c5a2 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:59.156943 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 506/987] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:18:59.164268 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a301a-ad90-4f78-bcd6-c692e5c11a4a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:59.164775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a301a-ad90-4f78-bcd6-c692e5c11a4a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:59.165267 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a301a-ad90-4f78-bcd6-c692e5c11a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:59.196719 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-ea3a301a-ad90-4f78-bcd6-c692e5c11a4a None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:59.198868 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ea3a301a-ad90-4f78-bcd6-c692e5c11a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:59.302542 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f2199-fda1-4af0-9c66-334df8d66b29 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:59.306770 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 507/988] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:59 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:18:59.325146 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:59.334529 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a301a-ad90-4f78-bcd6-c692e5c11a4a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:59.377240 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f041bd-7e9e-4fac-a866-c800ccd4c4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:59.378616 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f041bd-7e9e-4fac-a866-c800ccd4c4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:59.379084 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f041bd-7e9e-4fac-a866-c800ccd4c4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:59.379560 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f041bd-7e9e-4fac-a866-c800ccd4c4f2 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:59.381877 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f041bd-7e9e-4fac-a866-c800ccd4c4f2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:59.382501 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f041bd-7e9e-4fac-a866-c800ccd4c4f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:59.384508 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f041bd-7e9e-4fac-a866-c800ccd4c4f2 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:59.401812 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 482/989] 10.222.0.65 () {56 vars in 1122 bytes} [Thu May 19 21:18:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:59.403953 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f041bd-7e9e-4fac-a866-c800ccd4c4f2 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:59.499831 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:59.531029 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a301a-ad90-4f78-bcd6-c692e5c11a4a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:59.564644 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f041bd-7e9e-4fac-a866-c800ccd4c4f2 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:59.593679 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48c37b-57d7-40ec-9d3a-9890f558933b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:59.594646 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48c37b-57d7-40ec-9d3a-9890f558933b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:59.594978 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48c37b-57d7-40ec-9d3a-9890f558933b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:59.595231 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48c37b-57d7-40ec-9d3a-9890f558933b None admin] PATH_INFO: `/v3/services/image` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:59.595974 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f48c37b-57d7-40ec-9d3a-9890f558933b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:59.597451 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f48c37b-57d7-40ec-9d3a-9890f558933b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:59.611410 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-2f48c37b-57d7-40ec-9d3a-9890f558933b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 21:18:59.611410 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:59.611410 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:18:59.611410 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:18:59.611410 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:18:59.611410 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:18:59.611410 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:18:59.611410 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:18:59.611410 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 21:18:59.611410 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:59.611410 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:18:59.611410 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:59.611410 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:18:59.611410 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:18:59.611410 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:18:59.611410 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:18:59.611410 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:18:59.613163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:18:59.613163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:18:59.613163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:18:59.613163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:18:59.613163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:18:59.613163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:18:59.613163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:18:59.613163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:18:59.613163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:18:59.613163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:18:59.613163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:18:59.613163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:18:59.613163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:18:59.613163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:18:59.613163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:18:59.613163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:18:59.615181 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:18:59.615181 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:18:59.615181 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:18:59.615181 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:18:59.615181 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:18:59.615181 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:18:59.615181 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:18:59.615181 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:18:59.615181 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:18:59.615181 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:18:59.615181 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:18:59.615181 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:18:59.615181 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 21:18:59.615181 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:18:59.615181 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 508/990] 10.222.0.65 () {56 vars in 1105 bytes} [Thu May 19 21:18:59 2022] GET /identity/v3/services/image => generated 86 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:18:59.616310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d181b0-43a1-499d-8b83-99678e5fa6fe None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:59.616310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d181b0-43a1-499d-8b83-99678e5fa6fe None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:59.616310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d181b0-43a1-499d-8b83-99678e5fa6fe None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:59.617709 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 509/991] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:18:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:18:59.620664 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d181b0-43a1-499d-8b83-99678e5fa6fe None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:59.622701 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1301f8-314e-45cc-80e6-37a40a749caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:59.623879 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1301f8-314e-45cc-80e6-37a40a749caf None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:59.624407 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1301f8-314e-45cc-80e6-37a40a749caf None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:59.624897 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1301f8-314e-45cc-80e6-37a40a749caf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:59.626487 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1301f8-314e-45cc-80e6-37a40a749caf None admin] RBAC: Adding query filter params (interface=admin, service_id=88e7476037f94959a2974ed9dbf15390, region_id=RegionOne) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:59.627055 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1301f8-314e-45cc-80e6-37a40a749caf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:59.629925 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1301f8-314e-45cc-80e6-37a40a749caf None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:59.638212 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 483/992] 10.222.0.65 () {56 vars in 1249 bytes} [Thu May 19 21:18:59 2022] GET /identity/v3/endpoints?service_id=88e7476037f94959a2974ed9dbf15390&interface=admin®ion_id=RegionOne => generated 193 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:18:59.639682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafc0d6-7dc9-4d45-915b-3edc2d88770b None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:59.639968 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafc0d6-7dc9-4d45-915b-3edc2d88770b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:59.640251 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafc0d6-7dc9-4d45-915b-3edc2d88770b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:59.661853 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-9eafc0d6-7dc9-4d45-915b-3edc2d88770b None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:18:59.663362 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9eafc0d6-7dc9-4d45-915b-3edc2d88770b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:18:59.712757 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:59.773107 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 484/993] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:18:59 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:18:59.774499 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eafc0d6-7dc9-4d45-915b-3edc2d88770b None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:59.789585 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d181b0-43a1-499d-8b83-99678e5fa6fe None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:59.844368 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ffdc97-8500-4888-b045-85ae2837a2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:59.845456 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffdc97-8500-4888-b045-85ae2837a2c7 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:59.845734 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffdc97-8500-4888-b045-85ae2837a2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:59.846038 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffdc97-8500-4888-b045-85ae2837a2c7 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:59.846796 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ffdc97-8500-4888-b045-85ae2837a2c7 None admin] RBAC: Adding query filter params (name=image) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:18:59.847105 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ffdc97-8500-4888-b045-85ae2837a2c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:59.848632 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ffdc97-8500-4888-b045-85ae2837a2c7 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:59.867573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 510/994] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:18:59.867573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ffdc97-8500-4888-b045-85ae2837a2c7 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:59.881763 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:18:59.927574 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eafc0d6-7dc9-4d45-915b-3edc2d88770b None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:18:59.972218 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e310d8fb-66ad-4823-af62-7e67b57ae260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:18:59.972897 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310d8fb-66ad-4823-af62-7e67b57ae260 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:18:59.973603 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310d8fb-66ad-4823-af62-7e67b57ae260 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:18:59.973820 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310d8fb-66ad-4823-af62-7e67b57ae260 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:18:59.974962 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e310d8fb-66ad-4823-af62-7e67b57ae260 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:18:59.976399 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e310d8fb-66ad-4823-af62-7e67b57ae260 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:18:59.983305 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 485/995] 10.222.0.65 () {56 vars in 1087 bytes} [Thu May 19 21:18:59 2022] GET /identity/v3/services => generated 1727 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:18:59.984216 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e310d8fb-66ad-4823-af62-7e67b57ae260 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:18:59.991295 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ffdc97-8500-4888-b045-85ae2837a2c7 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:00.039848 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:00.052681 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f696be-1e4d-450e-a445-948f148494a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:00.053536 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f696be-1e4d-450e-a445-948f148494a8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:00.053790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f696be-1e4d-450e-a445-948f148494a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:00.054067 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f696be-1e4d-450e-a445-948f148494a8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:00.054883 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f696be-1e4d-450e-a445-948f148494a8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:00.056323 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f696be-1e4d-450e-a445-948f148494a8 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:00.072398 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-b2f696be-1e4d-450e-a445-948f148494a8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 21:19:00.072398 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:19:00.072398 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:19:00.072398 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:19:00.072398 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:19:00.072398 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:19:00.072398 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:19:00.072398 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:19:00.072398 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 21:19:00.072398 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:19:00.072398 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:19:00.072398 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:19:00.072398 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:19:00.072398 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:19:00.072398 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:19:00.072398 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:19:00.072398 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:19:00.073530 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:19:00.073530 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:19:00.073530 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:19:00.073530 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:19:00.073530 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:19:00.073530 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:19:00.073530 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:19:00.073530 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:19:00.073530 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:19:00.073530 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:19:00.073530 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:19:00.073530 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:19:00.073530 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:19:00.073530 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:19:00.073530 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:19:00.073530 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:19:00.074438 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:19:00.074438 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:19:00.074438 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:19:00.074438 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:19:00.074438 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:19:00.074438 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:19:00.074438 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:19:00.074438 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:19:00.074438 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:19:00.074438 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:19:00.074438 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:19:00.074438 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:19:00.074438 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 21:19:00.074438 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:19:00.074438 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 511/996] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:18:59 2022] GET /identity/v3/services/compute => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:19:00.084218 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f696be-1e4d-450e-a445-948f148494a8 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:00.196242 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e310d8fb-66ad-4823-af62-7e67b57ae260 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:00.220485 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:00.261724 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f696be-1e4d-450e-a445-948f148494a8 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:00.262309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a868b3a6-7376-41fa-bd95-c0bb67a9b316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:00.263815 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868b3a6-7376-41fa-bd95-c0bb67a9b316 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:00.263815 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868b3a6-7376-41fa-bd95-c0bb67a9b316 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:00.264117 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868b3a6-7376-41fa-bd95-c0bb67a9b316 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:00.265235 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a868b3a6-7376-41fa-bd95-c0bb67a9b316 None admin] RBAC: Adding query filter params (type=image) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:00.265726 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a868b3a6-7376-41fa-bd95-c0bb67a9b316 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:00.268479 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a868b3a6-7376-41fa-bd95-c0bb67a9b316 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:00.276538 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 486/997] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:18:59 2022] GET /identity/v3/services?type=image => generated 122 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:00.329345 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cda1aa-6a20-4751-8484-88e5bf600435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:00.330563 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cda1aa-6a20-4751-8484-88e5bf600435 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:00.331069 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cda1aa-6a20-4751-8484-88e5bf600435 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:00.331577 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cda1aa-6a20-4751-8484-88e5bf600435 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:00.333910 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cda1aa-6a20-4751-8484-88e5bf600435 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:00.334554 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cda1aa-6a20-4751-8484-88e5bf600435 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:00.337273 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cda1aa-6a20-4751-8484-88e5bf600435 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:00.349634 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 512/998] 10.222.0.65 () {56 vars in 1112 bytes} [Thu May 19 21:19:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:19:00.358655 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a868b3a6-7376-41fa-bd95-c0bb67a9b316 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:00.566487 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:00.579398 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a868b3a6-7376-41fa-bd95-c0bb67a9b316 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:00.656298 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c65d5-1847-47c4-867a-6c7fee7e6bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:00.657455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c65d5-1847-47c4-867a-6c7fee7e6bfc None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:00.657965 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c65d5-1847-47c4-867a-6c7fee7e6bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:00.658501 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c65d5-1847-47c4-867a-6c7fee7e6bfc None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:00.659827 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06c65d5-1847-47c4-867a-6c7fee7e6bfc None admin] RBAC: Adding query filter params (type=compute) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:00.660357 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06c65d5-1847-47c4-867a-6c7fee7e6bfc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:00.664155 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06c65d5-1847-47c4-867a-6c7fee7e6bfc None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:00.674550 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 487/999] 10.222.0.65 () {56 vars in 1112 bytes} [Thu May 19 21:19:00 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:00.685611 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cda1aa-6a20-4751-8484-88e5bf600435 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:00.899332 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cda1aa-6a20-4751-8484-88e5bf600435 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:00.968431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df40c074-d468-4336-b538-615640db6ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:00.969573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40c074-d468-4336-b538-615640db6ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:00.970070 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40c074-d468-4336-b538-615640db6ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:00.970588 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40c074-d468-4336-b538-615640db6ee4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:00.972115 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df40c074-d468-4336-b538-615640db6ee4 None admin] RBAC: Adding query filter params (interface=public, service_id=1e9bc7f13b1441c2ba0a260bd0937eb6, region_id=RegionOne) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:00.972676 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df40c074-d468-4336-b538-615640db6ee4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:00.976460 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df40c074-d468-4336-b538-615640db6ee4 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:00.990424 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 513/1000] 10.222.0.65 () {56 vars in 1251 bytes} [Thu May 19 21:19:00 2022] GET /identity/v3/endpoints?service_id=1e9bc7f13b1441c2ba0a260bd0937eb6&interface=public®ion_id=RegionOne => generated 194 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:00.996687 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c65d5-1847-47c4-867a-6c7fee7e6bfc None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:01.186393 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c65d5-1847-47c4-867a-6c7fee7e6bfc None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:01.253305 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a27d4e6-6341-4e8f-9723-c38b28cf966a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:01.254593 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27d4e6-6341-4e8f-9723-c38b28cf966a None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:01.255077 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27d4e6-6341-4e8f-9723-c38b28cf966a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:01.255520 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27d4e6-6341-4e8f-9723-c38b28cf966a None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:01.256820 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a27d4e6-6341-4e8f-9723-c38b28cf966a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:01.259417 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a27d4e6-6341-4e8f-9723-c38b28cf966a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:01.273212 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 488/1001] 10.222.0.65 () {56 vars in 1087 bytes} [Thu May 19 21:19:00 2022] GET /identity/v3/services => generated 1727 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:19:02.966481 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce488d13-186c-4ee4-b450-9cb5771e031e None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:02.967276 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce488d13-186c-4ee4-b450-9cb5771e031e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:02.967453 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce488d13-186c-4ee4-b450-9cb5771e031e None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:02.970304 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 514/1002] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:19:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:19:02.979531 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabd440-7fca-4d9e-bda4-e3722aaf1014 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:02.980292 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabd440-7fca-4d9e-bda4-e3722aaf1014 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:02.980461 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabd440-7fca-4d9e-bda4-e3722aaf1014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:03.010216 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-3fabd440-7fca-4d9e-bda4-e3722aaf1014 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:19:03.012638 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3fabd440-7fca-4d9e-bda4-e3722aaf1014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:19:03.050587 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:03.096902 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb6d006-09e2-4856-bb8e-0410799c3ab5 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:03.097330 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb6d006-09e2-4856-bb8e-0410799c3ab5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:03.097786 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb6d006-09e2-4856-bb8e-0410799c3ab5 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:03.100463 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 515/1003] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:19:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:19:03.108838 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e9727-b1cd-445b-a4fd-7833dde46ca1 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:03.109180 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e9727-b1cd-445b-a4fd-7833dde46ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:03.109628 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e9727-b1cd-445b-a4fd-7833dde46ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:03.128306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:03.138270 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-8a8e9727-b1cd-445b-a4fd-7833dde46ca1 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:19:03.140708 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8a8e9727-b1cd-445b-a4fd-7833dde46ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:19:03.146035 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 489/1004] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:19:02 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:19:03.184333 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fabd440-7fca-4d9e-bda4-e3722aaf1014 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:03.266174 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 516/1005] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:19:03 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:19:03.303422 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e9727-b1cd-445b-a4fd-7833dde46ca1 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:03.343561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fabd440-7fca-4d9e-bda4-e3722aaf1014 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:03.388165 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec107b8-5755-4c08-a677-a15c734cac99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:03.388898 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec107b8-5755-4c08-a677-a15c734cac99 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:03.389197 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec107b8-5755-4c08-a677-a15c734cac99 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:03.389480 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec107b8-5755-4c08-a677-a15c734cac99 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:03.390113 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec107b8-5755-4c08-a677-a15c734cac99 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:03.391546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec107b8-5755-4c08-a677-a15c734cac99 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:03.405155 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-eec107b8-5755-4c08-a677-a15c734cac99 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 21:19:03.405155 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:19:03.405155 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:19:03.405155 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:19:03.405155 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:19:03.405155 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:19:03.405155 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:19:03.405155 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:19:03.405155 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 21:19:03.405155 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:19:03.405155 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:19:03.405155 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:19:03.405155 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:19:03.405155 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:19:03.405155 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:19:03.405155 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:19:03.405155 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:19:03.407293 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:19:03.407293 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:19:03.407293 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:19:03.407293 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:19:03.407293 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:19:03.407293 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:19:03.407293 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:19:03.407293 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:19:03.407293 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:19:03.407293 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:19:03.407293 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:19:03.407293 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:19:03.407293 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:19:03.407293 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:19:03.407293 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:19:03.407293 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:19:03.409171 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:19:03.409171 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:19:03.409171 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:19:03.409171 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:19:03.409171 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:19:03.409171 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:19:03.409171 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:19:03.409171 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:19:03.409171 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:19:03.409171 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:19:03.409171 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:19:03.409171 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:19:03.409171 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 21:19:03.409171 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:19:03.412136 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 490/1006] 10.222.0.65 () {56 vars in 1126 bytes} [Thu May 19 21:19:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:19:03.413101 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec107b8-5755-4c08-a677-a15c734cac99 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:03.437397 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:03.497232 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e9727-b1cd-445b-a4fd-7833dde46ca1 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:03.554770 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe9fe22-31bc-4972-ac50-46f04dd94db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:03.555827 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe9fe22-31bc-4972-ac50-46f04dd94db0 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:03.556290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe9fe22-31bc-4972-ac50-46f04dd94db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:03.556746 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe9fe22-31bc-4972-ac50-46f04dd94db0 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:03.557808 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe9fe22-31bc-4972-ac50-46f04dd94db0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:03.560244 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe9fe22-31bc-4972-ac50-46f04dd94db0 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:03.565302 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:03.594313 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec107b8-5755-4c08-a677-a15c734cac99 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:03.599941 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 517/1007] 10.222.0.65 () {60 vars in 1141 bytes} [Thu May 19 21:19:03 2022] POST /identity/v3/services => generated 242 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:19:03.667989 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c669ad-8162-42e5-9f79-15111c2b9ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:03.668810 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c669ad-8162-42e5-9f79-15111c2b9ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:03.669309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c669ad-8162-42e5-9f79-15111c2b9ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:03.669838 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c669ad-8162-42e5-9f79-15111c2b9ce1 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:03.671166 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c669ad-8162-42e5-9f79-15111c2b9ce1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:03.671706 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c669ad-8162-42e5-9f79-15111c2b9ce1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:03.674635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c669ad-8162-42e5-9f79-15111c2b9ce1 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:03.684791 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 491/1008] 10.222.0.65 () {56 vars in 1122 bytes} [Thu May 19 21:19:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:03.694760 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe9fe22-31bc-4972-ac50-46f04dd94db0 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:03.894304 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe9fe22-31bc-4972-ac50-46f04dd94db0 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:03.963847 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f518f2-9a8e-4426-8328-7612d67470d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:03.964934 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f518f2-9a8e-4426-8328-7612d67470d2 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:03.965458 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f518f2-9a8e-4426-8328-7612d67470d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:03.965986 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f518f2-9a8e-4426-8328-7612d67470d2 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:03.967433 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f518f2-9a8e-4426-8328-7612d67470d2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:03.968374 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f518f2-9a8e-4426-8328-7612d67470d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:03.970326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f518f2-9a8e-4426-8328-7612d67470d2 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:03.981299 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 518/1009] 10.222.0.65 () {56 vars in 1122 bytes} [Thu May 19 21:19:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:03.987243 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c669ad-8162-42e5-9f79-15111c2b9ce1 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:04.157873 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:04.195448 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c669ad-8162-42e5-9f79-15111c2b9ce1 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:04.236315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db6bb1c-a34f-4be7-81ce-d7cc6cf077fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:04.236902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6bb1c-a34f-4be7-81ce-d7cc6cf077fa None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:04.237184 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6bb1c-a34f-4be7-81ce-d7cc6cf077fa None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:04.237458 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6bb1c-a34f-4be7-81ce-d7cc6cf077fa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:04.238223 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db6bb1c-a34f-4be7-81ce-d7cc6cf077fa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:04.240046 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db6bb1c-a34f-4be7-81ce-d7cc6cf077fa None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:04.266297 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 492/1010] 10.222.0.65 () {60 vars in 1144 bytes} [Thu May 19 21:19:03 2022] POST /identity/v3/endpoints => generated 325 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:19:04.266650 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6f742-7032-4e30-9060-a020553dc24f None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:04.266650 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6f742-7032-4e30-9060-a020553dc24f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:04.266650 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6f742-7032-4e30-9060-a020553dc24f None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:04.267226 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 519/1011] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:19:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:19:04.274717 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e54ec7-a45a-44f6-a50a-5bbf9a288465 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:04.274993 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e54ec7-a45a-44f6-a50a-5bbf9a288465 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:04.275342 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e54ec7-a45a-44f6-a50a-5bbf9a288465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:04.303676 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-64e54ec7-a45a-44f6-a50a-5bbf9a288465 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:19:04.304942 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-64e54ec7-a45a-44f6-a50a-5bbf9a288465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:19:04.406282 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 493/1012] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:19:04 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:19:04.445667 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de6f742-7032-4e30-9060-a020553dc24f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:04.543816 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:04.625171 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de6f742-7032-4e30-9060-a020553dc24f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:04.689465 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6649af8a-2fad-4e83-aac6-7d16e6f2813b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:04.690902 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649af8a-2fad-4e83-aac6-7d16e6f2813b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:04.691393 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649af8a-2fad-4e83-aac6-7d16e6f2813b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:04.692054 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649af8a-2fad-4e83-aac6-7d16e6f2813b None admin] PATH_INFO: `/v3/services/compute` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:04.693320 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6649af8a-2fad-4e83-aac6-7d16e6f2813b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:04.696063 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6649af8a-2fad-4e83-aac6-7d16e6f2813b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:04.719538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-6649af8a-2fad-4e83-aac6-7d16e6f2813b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 21:19:04.719538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:19:04.719538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:19:04.719538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:19:04.719538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:19:04.719538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:19:04.719538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:19:04.719538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:19:04.719538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 21:19:04.719538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:19:04.719538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:19:04.719538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:19:04.719538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:19:04.719538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:19:04.719538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:19:04.719538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:19:04.719538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:19:04.720962 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:19:04.720962 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:19:04.720962 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:19:04.720962 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:19:04.720962 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:19:04.720962 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:19:04.720962 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:19:04.720962 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:19:04.720962 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:19:04.720962 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:19:04.720962 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:19:04.720962 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:19:04.720962 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:19:04.720962 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:19:04.720962 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:19:04.720962 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:19:04.722257 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:19:04.722257 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:19:04.722257 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:19:04.722257 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:19:04.722257 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:19:04.722257 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:19:04.722257 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:19:04.722257 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:19:04.722257 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:19:04.722257 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:19:04.722257 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:19:04.722257 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:19:04.722257 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 21:19:04.722257 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:19:04.723996 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 520/1013] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:19:04 2022] GET /identity/v3/services/compute => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:19:04.730283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e54ec7-a45a-44f6-a50a-5bbf9a288465 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:04.841531 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:04.902378 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e54ec7-a45a-44f6-a50a-5bbf9a288465 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:04.947040 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb476fe-5929-4a9c-88cc-56a28f739a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:04.947727 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb476fe-5929-4a9c-88cc-56a28f739a9b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:04.948013 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb476fe-5929-4a9c-88cc-56a28f739a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:04.948280 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb476fe-5929-4a9c-88cc-56a28f739a9b None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:04.949006 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb476fe-5929-4a9c-88cc-56a28f739a9b None admin] RBAC: Adding query filter params (name=compute) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:04.949310 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb476fe-5929-4a9c-88cc-56a28f739a9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:04.950764 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb476fe-5929-4a9c-88cc-56a28f739a9b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:04.956668 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 494/1014] 10.222.0.65 () {56 vars in 1112 bytes} [Thu May 19 21:19:04 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:04.962706 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6649af8a-2fad-4e83-aac6-7d16e6f2813b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:05.096030 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6649af8a-2fad-4e83-aac6-7d16e6f2813b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:05.125112 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:05.138197 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc8079b-c9f9-4966-9229-98e308fa6d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:05.138927 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8079b-c9f9-4966-9229-98e308fa6d94 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:05.139208 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8079b-c9f9-4966-9229-98e308fa6d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:05.139487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc8079b-c9f9-4966-9229-98e308fa6d94 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:05.140225 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc8079b-c9f9-4966-9229-98e308fa6d94 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:05.140532 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc8079b-c9f9-4966-9229-98e308fa6d94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:05.141989 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc8079b-c9f9-4966-9229-98e308fa6d94 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:05.148620 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 521/1015] 10.222.0.65 () {56 vars in 1112 bytes} [Thu May 19 21:19:04 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:05.154447 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb476fe-5929-4a9c-88cc-56a28f739a9b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:05.288834 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb476fe-5929-4a9c-88cc-56a28f739a9b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:05.335062 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:05.348971 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617158c-f9d9-48c3-a449-6d87b4501184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:05.349922 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617158c-f9d9-48c3-a449-6d87b4501184 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:05.350414 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617158c-f9d9-48c3-a449-6d87b4501184 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:05.350874 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617158c-f9d9-48c3-a449-6d87b4501184 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:05.351681 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b617158c-f9d9-48c3-a449-6d87b4501184 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:05.353205 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b617158c-f9d9-48c3-a449-6d87b4501184 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:05.387097 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 495/1016] 10.222.0.65 () {60 vars in 1144 bytes} [Thu May 19 21:19:05 2022] POST /identity/v3/endpoints => generated 334 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:19:06.015046 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5a92f-8e7f-49ab-9013-6510072d18ce None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:06.015329 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5a92f-8e7f-49ab-9013-6510072d18ce None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:06.015604 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5a92f-8e7f-49ab-9013-6510072d18ce None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:06.017005 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 522/1017] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:19:06.022763 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f542b-6d3d-4317-baa3-d4fa1691363e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:06.023050 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f542b-6d3d-4317-baa3-d4fa1691363e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:06.023328 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f542b-6d3d-4317-baa3-d4fa1691363e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:06.040900 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-8e7f542b-6d3d-4317-baa3-d4fa1691363e None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:19:06.042090 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8e7f542b-6d3d-4317-baa3-d4fa1691363e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:19:06.144031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 496/1018] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:19:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:19:06.166508 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c5a92f-8e7f-49ab-9013-6510072d18ce None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:06.294558 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c5a92f-8e7f-49ab-9013-6510072d18ce None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:06.337003 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea90262-76c9-4b39-80c9-4b2f7a7bd5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:06.337860 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea90262-76c9-4b39-80c9-4b2f7a7bd5b9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:06.338169 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea90262-76c9-4b39-80c9-4b2f7a7bd5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:06.338560 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea90262-76c9-4b39-80c9-4b2f7a7bd5b9 None admin] PATH_INFO: `/v3/services/image` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:06.339277 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea90262-76c9-4b39-80c9-4b2f7a7bd5b9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:06.340762 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea90262-76c9-4b39-80c9-4b2f7a7bd5b9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:06.363188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-6ea90262-76c9-4b39-80c9-4b2f7a7bd5b9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 21:19:06.363188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:19:06.363188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:19:06.363188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:19:06.363188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:19:06.363188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:19:06.363188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:19:06.363188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:19:06.363188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 21:19:06.363188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:19:06.363188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:19:06.363188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:19:06.363188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:19:06.363188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:19:06.363188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:19:06.363188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:19:06.363188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:19:06.365141 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:19:06.365141 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:19:06.365141 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:19:06.365141 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:19:06.365141 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:19:06.365141 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:19:06.365141 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:19:06.365141 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:19:06.365141 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:19:06.365141 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:19:06.365141 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:19:06.365141 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:19:06.365141 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:19:06.365141 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:19:06.365141 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:19:06.365141 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:19:06.366223 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:19:06.366223 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:19:06.366223 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:19:06.366223 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:19:06.366223 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:19:06.366223 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:19:06.366223 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:19:06.366223 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:19:06.366223 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:19:06.366223 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:19:06.366223 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:19:06.366223 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:19:06.366223 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 21:19:06.366223 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:19:06.369013 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 523/1019] 10.222.0.65 () {56 vars in 1105 bytes} [Thu May 19 21:19:06 2022] GET /identity/v3/services/image => generated 86 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:19:06.374936 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f542b-6d3d-4317-baa3-d4fa1691363e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:06.498034 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f542b-6d3d-4317-baa3-d4fa1691363e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:06.511571 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:06.542816 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec35a2d-d902-4f36-8f7a-3aa00794dfb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:06.543498 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec35a2d-d902-4f36-8f7a-3aa00794dfb1 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:06.543773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec35a2d-d902-4f36-8f7a-3aa00794dfb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:06.544038 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec35a2d-d902-4f36-8f7a-3aa00794dfb1 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:06.544745 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec35a2d-d902-4f36-8f7a-3aa00794dfb1 None admin] RBAC: Adding query filter params (name=image) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:06.545045 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec35a2d-d902-4f36-8f7a-3aa00794dfb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:06.546575 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec35a2d-d902-4f36-8f7a-3aa00794dfb1 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:06.556786 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 497/1020] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:19:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:19:06.566119 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea90262-76c9-4b39-80c9-4b2f7a7bd5b9 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:06.636836 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:06.713196 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea90262-76c9-4b39-80c9-4b2f7a7bd5b9 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:06.780037 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b80ed22-8902-4fe8-b0d9-5a8ef12d266f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:06.781164 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80ed22-8902-4fe8-b0d9-5a8ef12d266f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:06.781674 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80ed22-8902-4fe8-b0d9-5a8ef12d266f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:06.781965 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80ed22-8902-4fe8-b0d9-5a8ef12d266f None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:06.783405 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b80ed22-8902-4fe8-b0d9-5a8ef12d266f None admin] RBAC: Adding query filter params (type=image) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:06.783960 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b80ed22-8902-4fe8-b0d9-5a8ef12d266f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:06.786758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b80ed22-8902-4fe8-b0d9-5a8ef12d266f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:06.802305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 524/1021] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:19:06 2022] GET /identity/v3/services?type=image => generated 350 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:06.806163 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec35a2d-d902-4f36-8f7a-3aa00794dfb1 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:06.816483 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e06a11-0500-41f4-90e9-d1c6a97e9f37 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:06.816483 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e06a11-0500-41f4-90e9-d1c6a97e9f37 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:06.818131 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e06a11-0500-41f4-90e9-d1c6a97e9f37 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:06.820734 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 525/1022] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:19:06.825273 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d7a25-a348-4f50-a7b4-61e1d0554b83 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:06.825612 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d7a25-a348-4f50-a7b4-61e1d0554b83 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:06.825839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d7a25-a348-4f50-a7b4-61e1d0554b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:06.851173 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-276d7a25-a348-4f50-a7b4-61e1d0554b83 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:19:06.853888 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-276d7a25-a348-4f50-a7b4-61e1d0554b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:19:06.960060 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec35a2d-d902-4f36-8f7a-3aa00794dfb1 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:06.961875 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 526/1023] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:19:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:19:06.985816 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d7a25-a348-4f50-a7b4-61e1d0554b83 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:07.012105 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09014938-0c38-44cc-8a69-34f7992b6326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:07.013251 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09014938-0c38-44cc-8a69-34f7992b6326 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:07.013740 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09014938-0c38-44cc-8a69-34f7992b6326 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:07.014225 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09014938-0c38-44cc-8a69-34f7992b6326 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:07.015811 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09014938-0c38-44cc-8a69-34f7992b6326 None admin] RBAC: Adding query filter params (interface=public, service_id=ed928906886d4cae912779d23c6446c5, region_id=RegionOne) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:07.016375 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09014938-0c38-44cc-8a69-34f7992b6326 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:07.018738 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09014938-0c38-44cc-8a69-34f7992b6326 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:07.028931 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 498/1024] 10.222.0.65 () {56 vars in 1251 bytes} [Thu May 19 21:19:06 2022] GET /identity/v3/endpoints?service_id=ed928906886d4cae912779d23c6446c5&interface=public®ion_id=RegionOne => generated 194 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:07.036325 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09014938-0c38-44cc-8a69-34f7992b6326 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:07.045093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:07.085920 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:07.152387 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d7a25-a348-4f50-a7b4-61e1d0554b83 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:07.162203 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09014938-0c38-44cc-8a69-34f7992b6326 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:07.210935 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99939615-eedf-49f2-8679-0ee64ad05742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:07.212054 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99939615-eedf-49f2-8679-0ee64ad05742 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:07.212205 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38222e-7703-4232-a5ea-103072028e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:07.213045 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38222e-7703-4232-a5ea-103072028e57 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:07.213248 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99939615-eedf-49f2-8679-0ee64ad05742 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:07.213357 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38222e-7703-4232-a5ea-103072028e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:07.213656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38222e-7703-4232-a5ea-103072028e57 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:07.213906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99939615-eedf-49f2-8679-0ee64ad05742 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:07.214973 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99939615-eedf-49f2-8679-0ee64ad05742 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:07.215402 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb38222e-7703-4232-a5ea-103072028e57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:07.216839 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99939615-eedf-49f2-8679-0ee64ad05742 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:07.217019 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb38222e-7703-4232-a5ea-103072028e57 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:07.224568 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 527/1025] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:19:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:19:07.227402 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38222e-7703-4232-a5ea-103072028e57 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:07.232612 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 499/1026] 10.222.0.65 () {56 vars in 1087 bytes} [Thu May 19 21:19:07 2022] GET /identity/v3/services => generated 1957 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 21:19:07.388124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38222e-7703-4232-a5ea-103072028e57 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:07.410096 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:07.438677 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0e742f-4a9b-48c9-b058-f7e28e1d4812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:07.439782 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e742f-4a9b-48c9-b058-f7e28e1d4812 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:07.440248 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e742f-4a9b-48c9-b058-f7e28e1d4812 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:07.440546 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e742f-4a9b-48c9-b058-f7e28e1d4812 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:07.441330 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0e742f-4a9b-48c9-b058-f7e28e1d4812 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:07.443339 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0e742f-4a9b-48c9-b058-f7e28e1d4812 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:07.473591 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 528/1027] 10.222.0.65 () {60 vars in 1141 bytes} [Thu May 19 21:19:07 2022] POST /identity/v3/projects => generated 311 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:19:07.986475 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db0d7d-982d-48e5-bb72-1e7acc196255 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:07.987208 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db0d7d-982d-48e5-bb72-1e7acc196255 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:07.987208 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db0d7d-982d-48e5-bb72-1e7acc196255 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:07.988555 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 500/1028] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:19:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:19:07.994736 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd720fd1-6795-471f-a43d-4ae004710540 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:07.995134 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd720fd1-6795-471f-a43d-4ae004710540 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:07.995361 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd720fd1-6795-471f-a43d-4ae004710540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:08.015089 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-bd720fd1-6795-471f-a43d-4ae004710540 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:19:08.016836 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-bd720fd1-6795-471f-a43d-4ae004710540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:19:08.144429 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 529/1029] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:19:07 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:19:08.192660 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db0d7d-982d-48e5-bb72-1e7acc196255 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:08.325637 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db0d7d-982d-48e5-bb72-1e7acc196255 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:08.327594 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:08.370457 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ced6e-7568-4ab5-8e57-eba5f1ee1ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:08.371336 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ced6e-7568-4ab5-8e57-eba5f1ee1ceb None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:08.371630 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ced6e-7568-4ab5-8e57-eba5f1ee1ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:08.371887 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ced6e-7568-4ab5-8e57-eba5f1ee1ceb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:08.384991 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8ced6e-7568-4ab5-8e57-eba5f1ee1ceb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:19:08.393387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-da8ced6e-7568-4ab5-8e57-eba5f1ee1ceb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 21:19:08.393387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:19:08.393387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:19:08.393387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:19:08.393387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:19:08.393387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:19:08.393387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:19:08.393387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:19:08.393387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:19:08.393387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:19:08.393387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:19:08.393387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:19:08.393387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:19:08.393387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:19:08.393387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:19:08.393387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:19:08.395639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:19:08.395639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:19:08.395639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:19:08.395639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:19:08.395639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:19:08.395639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:19:08.395639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:19:08.395639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:19:08.395639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:19:08.395639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:19:08.395639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:19:08.395639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:19:08.395639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:19:08.395639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:19:08.395639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:19:08.395639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:19:08.397431 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:19:08.397431 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:19:08.397431 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:19:08.397431 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:19:08.397431 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:19:08.397431 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:19:08.397431 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:19:08.397431 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:19:08.397431 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:19:08.397431 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:19:08.397431 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 21:19:08.397431 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:19:08.397431 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 501/1030] 10.222.0.65 () {56 vars in 1120 bytes} [Thu May 19 21:19:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:19:08.404358 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd720fd1-6795-471f-a43d-4ae004710540 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:08.583573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:08.596654 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd720fd1-6795-471f-a43d-4ae004710540 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:08.661105 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d3f202-dde3-4c7b-8af2-546eb222fd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:08.662390 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3f202-dde3-4c7b-8af2-546eb222fd39 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:08.662872 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3f202-dde3-4c7b-8af2-546eb222fd39 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:08.663340 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3f202-dde3-4c7b-8af2-546eb222fd39 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:08.664725 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d3f202-dde3-4c7b-8af2-546eb222fd39 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:08.665253 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d3f202-dde3-4c7b-8af2-546eb222fd39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:08.669995 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d3f202-dde3-4c7b-8af2-546eb222fd39 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:08.698288 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 530/1031] 10.222.0.65 () {56 vars in 1115 bytes} [Thu May 19 21:19:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:08.703432 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ced6e-7568-4ab5-8e57-eba5f1ee1ceb None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:08.903592 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ced6e-7568-4ab5-8e57-eba5f1ee1ceb None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:08.964728 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b566d71-9044-411a-b5e1-e6ad8a6f67ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:08.965992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b566d71-9044-411a-b5e1-e6ad8a6f67ad None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:08.966708 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b566d71-9044-411a-b5e1-e6ad8a6f67ad None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:08.967188 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b566d71-9044-411a-b5e1-e6ad8a6f67ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:08.995763 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b566d71-9044-411a-b5e1-e6ad8a6f67ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:08.998459 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b566d71-9044-411a-b5e1-e6ad8a6f67ad None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:09.034377 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-7b566d71-9044-411a-b5e1-e6ad8a6f67ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:19:09.034377 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:19:09.034377 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:19:09.034377 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:19:09.034377 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:19:09.034377 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:19:09.034377 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:19:09.034377 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:19:09.034377 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:19:09.034377 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:19:09.034377 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:19:09.034377 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:19:09.034377 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:19:09.034377 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:19:09.034377 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:19:09.036079 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:19:09.036079 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:19:09.036079 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:19:09.036079 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:19:09.036079 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:19:09.036079 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:19:09.036079 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:19:09.036079 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:19:09.036079 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:19:09.036079 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:19:09.036079 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:19:09.036079 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:19:09.036079 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:19:09.036079 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:19:09.036079 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:19:09.036079 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:19:09.036079 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:19:09.043950 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:19:09.043950 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:19:09.043950 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:19:09.043950 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:19:09.043950 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 502/1032] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:19:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:19:09.049011 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d3f202-dde3-4c7b-8af2-546eb222fd39 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:09.253750 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d3f202-dde3-4c7b-8af2-546eb222fd39 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:09.321032 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b6cfe8-4741-462e-82b4-eec8e9e1f06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:09.322185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b6cfe8-4741-462e-82b4-eec8e9e1f06b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:09.322707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b6cfe8-4741-462e-82b4-eec8e9e1f06b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:09.323197 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b6cfe8-4741-462e-82b4-eec8e9e1f06b None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:09.324577 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b6cfe8-4741-462e-82b4-eec8e9e1f06b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:09.325135 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b6cfe8-4741-462e-82b4-eec8e9e1f06b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:09.327685 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b6cfe8-4741-462e-82b4-eec8e9e1f06b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:09.358351 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 531/1033] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:19:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:09.361601 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b566d71-9044-411a-b5e1-e6ad8a6f67ad None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:09.396632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:09.542764 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b566d71-9044-411a-b5e1-e6ad8a6f67ad None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:09.585183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58247b3e-52e9-4e7e-ba56-56ca84290a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:09.586182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58247b3e-52e9-4e7e-ba56-56ca84290a75 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:09.586627 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58247b3e-52e9-4e7e-ba56-56ca84290a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:09.587033 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58247b3e-52e9-4e7e-ba56-56ca84290a75 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:09.596353 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58247b3e-52e9-4e7e-ba56-56ca84290a75 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:09.598047 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58247b3e-52e9-4e7e-ba56-56ca84290a75 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:09.615351 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-58247b3e-52e9-4e7e-ba56-56ca84290a75 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 21:19:09.615351 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:19:09.615351 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:19:09.615351 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:19:09.615351 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:19:09.615351 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:19:09.615351 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:19:09.615351 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:19:09.615351 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:19:09.615351 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:19:09.615351 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:19:09.615351 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:19:09.615351 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:19:09.615351 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:19:09.615351 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:19:09.615351 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:19:09.617239 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:19:09.617239 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:19:09.617239 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:19:09.617239 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:19:09.617239 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:19:09.617239 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:19:09.617239 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:19:09.617239 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:19:09.617239 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:19:09.617239 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:19:09.617239 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:19:09.617239 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:19:09.617239 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:19:09.617239 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:19:09.617239 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:19:09.617239 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:19:09.617239 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:19:09.617239 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:19:09.618585 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:19:09.618585 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:19:09.618585 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:19:09.618585 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:19:09.618585 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:19:09.618585 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:19:09.618585 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:19:09.618585 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:19:09.618585 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 21:19:09.618585 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:19:09.618585 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 503/1034] 10.222.0.65 () {56 vars in 1093 bytes} [Thu May 19 21:19:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:19:09.625505 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b6cfe8-4741-462e-82b4-eec8e9e1f06b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:09.649805 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:09.746326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b6cfe8-4741-462e-82b4-eec8e9e1f06b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:09.788214 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630ca776-7cc5-4bac-8edd-2653c18c292b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:09.788915 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ca776-7cc5-4bac-8edd-2653c18c292b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:09.789273 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ca776-7cc5-4bac-8edd-2653c18c292b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:09.789510 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ca776-7cc5-4bac-8edd-2653c18c292b None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:09.790373 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630ca776-7cc5-4bac-8edd-2653c18c292b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:09.790686 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630ca776-7cc5-4bac-8edd-2653c18c292b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:09.792153 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630ca776-7cc5-4bac-8edd-2653c18c292b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:09.813832 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 532/1035] 10.222.0.65 () {56 vars in 1133 bytes} [Thu May 19 21:19:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:09.820599 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58247b3e-52e9-4e7e-ba56-56ca84290a75 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:09.984781 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:10.026410 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58247b3e-52e9-4e7e-ba56-56ca84290a75 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:10.087648 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8918b82-397e-4ff6-98cc-66322c0d4551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:10.088666 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8918b82-397e-4ff6-98cc-66322c0d4551 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:10.089141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8918b82-397e-4ff6-98cc-66322c0d4551 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:10.089640 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8918b82-397e-4ff6-98cc-66322c0d4551 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:10.187189 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2086fafc-ff3c-4750-a39a-7bc206a6aa19 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:10.188200 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2086fafc-ff3c-4750-a39a-7bc206a6aa19 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:10.188582 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2086fafc-ff3c-4750-a39a-7bc206a6aa19 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:10.192003 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 533/1036] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:19:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:19:10.200771 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c1058-6a6f-425f-8083-0b99bd9d5be0 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:10.201278 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c1058-6a6f-425f-8083-0b99bd9d5be0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:10.201782 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c1058-6a6f-425f-8083-0b99bd9d5be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:10.221327 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8918b82-397e-4ff6-98cc-66322c0d4551 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:10.221995 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e33c1058-6a6f-425f-8083-0b99bd9d5be0 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:19:10.223960 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e33c1058-6a6f-425f-8083-0b99bd9d5be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:19:10.224605 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8918b82-397e-4ff6-98cc-66322c0d4551 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:10.315281 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 534/1037] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:19:10 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:19:10.316808 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4658adba-e8c8-4f2e-b17f-81be05412b13 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:10.317265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4658adba-e8c8-4f2e-b17f-81be05412b13 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:10.317794 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4658adba-e8c8-4f2e-b17f-81be05412b13 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:10.319607 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 535/1038] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:19:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:19:10.325912 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcfca0b-387a-4134-8060-ad8b3a54d463 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:10.326789 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcfca0b-387a-4134-8060-ad8b3a54d463 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:10.327197 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcfca0b-387a-4134-8060-ad8b3a54d463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:10.342408 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:10.350841 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9fcfca0b-387a-4134-8060-ad8b3a54d463 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:19:10.352051 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9fcfca0b-387a-4134-8060-ad8b3a54d463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:19:10.359292 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:10.409384 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 504/1039] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:19:09 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 21:19:10.411153 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8918b82-397e-4ff6-98cc-66322c0d4551 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:10.436742 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 536/1040] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:19:10 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:19:10.438410 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcfca0b-387a-4134-8060-ad8b3a54d463 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:10.564428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcfca0b-387a-4134-8060-ad8b3a54d463 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:10.588221 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8918b82-397e-4ff6-98cc-66322c0d4551 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:10.611115 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb13d680-af2d-4fe6-833e-a2c7ec9dc9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:10.612823 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb13d680-af2d-4fe6-833e-a2c7ec9dc9b8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:10.613115 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb13d680-af2d-4fe6-833e-a2c7ec9dc9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:10.613467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb13d680-af2d-4fe6-833e-a2c7ec9dc9b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:10.617376 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:10.627689 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb13d680-af2d-4fe6-833e-a2c7ec9dc9b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:10.629367 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb13d680-af2d-4fe6-833e-a2c7ec9dc9b8 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:10.648067 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f18a53-fbda-4e3a-89d8-c3b39d992a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:10.649082 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f18a53-fbda-4e3a-89d8-c3b39d992a56 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:10.649433 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f18a53-fbda-4e3a-89d8-c3b39d992a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:10.649930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f18a53-fbda-4e3a-89d8-c3b39d992a56 None admin] PATH_INFO: `/v3/services/image` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:10.651191 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f18a53-fbda-4e3a-89d8-c3b39d992a56 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:10.652471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-cb13d680-af2d-4fe6-833e-a2c7ec9dc9b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:19:10.652471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:19:10.652471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:19:10.652471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:19:10.652471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:19:10.652471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:19:10.652471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:19:10.652471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:19:10.652471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:19:10.652471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:19:10.652471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:19:10.652471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:19:10.652471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:19:10.652471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:19:10.652471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:19:10.654308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:19:10.654308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:19:10.654308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:19:10.654308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:19:10.654308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:19:10.654308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:19:10.654308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:19:10.654308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:19:10.654308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:19:10.654308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:19:10.654308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:19:10.654308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:19:10.654308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:19:10.654308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:19:10.654308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:19:10.654308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:19:10.654308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:19:10.656033 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:19:10.656033 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:19:10.656033 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:19:10.656033 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:19:10.656469 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f18a53-fbda-4e3a-89d8-c3b39d992a56 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:10.656636 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 537/1041] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:19:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:19:10.656636 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb13d680-af2d-4fe6-833e-a2c7ec9dc9b8 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:10.675674 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-10f18a53-fbda-4e3a-89d8-c3b39d992a56 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 21:19:10.675674 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:19:10.675674 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:19:10.675674 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:19:10.675674 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:19:10.675674 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:19:10.675674 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:19:10.675674 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:19:10.675674 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 21:19:10.675674 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:19:10.675674 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:19:10.675674 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:19:10.675674 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:19:10.675674 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:19:10.675674 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:19:10.675674 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:19:10.675674 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:19:10.677313 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:19:10.677313 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:19:10.677313 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:19:10.677313 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:19:10.677313 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:19:10.677313 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:19:10.677313 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:19:10.677313 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:19:10.677313 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:19:10.677313 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:19:10.677313 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:19:10.677313 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:19:10.677313 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:19:10.677313 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:19:10.677313 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:19:10.677313 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:19:10.679172 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:19:10.679172 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:19:10.679172 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:19:10.679172 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:19:10.679172 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:19:10.679172 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:19:10.679172 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:19:10.679172 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:19:10.679172 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:19:10.679172 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:19:10.679172 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:19:10.679172 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:19:10.679172 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 21:19:10.679172 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:19:10.679172 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 505/1042] 10.222.0.65 () {56 vars in 1105 bytes} [Thu May 19 21:19:10 2022] GET /identity/v3/services/image => generated 86 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:19:10.681772 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f18a53-fbda-4e3a-89d8-c3b39d992a56 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:10.738024 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:10.828598 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f18a53-fbda-4e3a-89d8-c3b39d992a56 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:10.842024 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb13d680-af2d-4fe6-833e-a2c7ec9dc9b8 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:10.886658 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c32ff-ad89-4f1a-9082-1418a82567d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:10.887240 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c32ff-ad89-4f1a-9082-1418a82567d4 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:10.887523 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c32ff-ad89-4f1a-9082-1418a82567d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:10.887800 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c32ff-ad89-4f1a-9082-1418a82567d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:10.888555 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359c32ff-ad89-4f1a-9082-1418a82567d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:10.888864 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359c32ff-ad89-4f1a-9082-1418a82567d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:10.890444 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359c32ff-ad89-4f1a-9082-1418a82567d4 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:10.907622 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb40bd-512d-45b0-9bfd-ec70d3f40923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:10.908162 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 506/1043] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:19:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:10.908451 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb40bd-512d-45b0-9bfd-ec70d3f40923 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:10.908735 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb40bd-512d-45b0-9bfd-ec70d3f40923 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:10.909032 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb40bd-512d-45b0-9bfd-ec70d3f40923 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:10.909704 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c32ff-ad89-4f1a-9082-1418a82567d4 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:10.910713 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeb40bd-512d-45b0-9bfd-ec70d3f40923 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:10.913155 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeb40bd-512d-45b0-9bfd-ec70d3f40923 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:10.915167 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 538/1044] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:19:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:10.918661 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb40bd-512d-45b0-9bfd-ec70d3f40923 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:10.990175 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:11.052008 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:11.059865 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c32ff-ad89-4f1a-9082-1418a82567d4 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:11.062821 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb40bd-512d-45b0-9bfd-ec70d3f40923 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:11.114299 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b03cc-b876-4961-883a-1c44a3ca7e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:11.115650 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b03cc-b876-4961-883a-1c44a3ca7e7f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:11.116118 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b03cc-b876-4961-883a-1c44a3ca7e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:11.116584 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b03cc-b876-4961-883a-1c44a3ca7e7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:11.125757 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb6fc65-a7e0-4d33-8817-41f4b41ed885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:11.127161 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09b03cc-b876-4961-883a-1c44a3ca7e7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:11.127371 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6fc65-a7e0-4d33-8817-41f4b41ed885 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:11.127756 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6fc65-a7e0-4d33-8817-41f4b41ed885 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:11.128236 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6fc65-a7e0-4d33-8817-41f4b41ed885 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:11.129043 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09b03cc-b876-4961-883a-1c44a3ca7e7f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:11.129588 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb6fc65-a7e0-4d33-8817-41f4b41ed885 None admin] RBAC: Adding query filter params (name=image) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:11.130121 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb6fc65-a7e0-4d33-8817-41f4b41ed885 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:11.132776 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb6fc65-a7e0-4d33-8817-41f4b41ed885 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:11.148506 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 507/1045] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:19:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:11.148506 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb6fc65-a7e0-4d33-8817-41f4b41ed885 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:11.154421 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-a09b03cc-b876-4961-883a-1c44a3ca7e7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:19:11.154421 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:19:11.154421 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:19:11.154421 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:19:11.154421 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:19:11.154421 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:19:11.154421 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:19:11.154421 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:19:11.154421 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:19:11.154421 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:19:11.154421 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:19:11.154421 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:19:11.154421 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:19:11.154421 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:19:11.154421 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:19:11.156224 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:19:11.156224 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:19:11.156224 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:19:11.156224 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:19:11.156224 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:19:11.156224 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:19:11.156224 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:19:11.156224 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:19:11.156224 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:19:11.156224 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:19:11.156224 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:19:11.156224 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:19:11.156224 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:19:11.156224 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:19:11.156224 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:19:11.156224 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:19:11.156224 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:19:11.158147 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:19:11.158147 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:19:11.158147 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:19:11.158147 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:19:11.158147 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:19:11.158147 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:19:11.158147 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:19:11.158147 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:19:11.158147 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 539/1046] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:19:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:19:11.158147 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b03cc-b876-4961-883a-1c44a3ca7e7f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:11.297908 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb6fc65-a7e0-4d33-8817-41f4b41ed885 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:11.336657 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b03cc-b876-4961-883a-1c44a3ca7e7f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:11.354629 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545003b-3b7a-43d9-a77a-108300553341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:11.355506 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545003b-3b7a-43d9-a77a-108300553341 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:11.355913 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545003b-3b7a-43d9-a77a-108300553341 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:11.356336 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545003b-3b7a-43d9-a77a-108300553341 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:11.357308 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9545003b-3b7a-43d9-a77a-108300553341 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:11.358988 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9545003b-3b7a-43d9-a77a-108300553341 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:11.360743 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:11.390189 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d4e49-5284-488c-b2ff-f630b146e470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:11.390949 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d4e49-5284-488c-b2ff-f630b146e470 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:11.391251 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d4e49-5284-488c-b2ff-f630b146e470 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:11.391596 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 508/1047] 10.222.0.65 () {60 vars in 1132 bytes} [Thu May 19 21:19:11 2022] POST /identity/v3/users => generated 285 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:19:11.391780 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d4e49-5284-488c-b2ff-f630b146e470 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:11.392630 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d4e49-5284-488c-b2ff-f630b146e470 None admin] RBAC: Adding query filter params (type=image) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:11.392942 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545003b-3b7a-43d9-a77a-108300553341 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:11.393099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d4e49-5284-488c-b2ff-f630b146e470 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:11.394422 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d4e49-5284-488c-b2ff-f630b146e470 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:11.412276 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 540/1048] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:19:11 2022] GET /identity/v3/services?type=image => generated 350 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:11.419318 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d4e49-5284-488c-b2ff-f630b146e470 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:11.433087 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:11.580922 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545003b-3b7a-43d9-a77a-108300553341 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:11.627818 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d4e49-5284-488c-b2ff-f630b146e470 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:11.638225 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b9381-c688-47f8-8438-54105ff71bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:11.638968 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b9381-c688-47f8-8438-54105ff71bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:11.639266 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b9381-c688-47f8-8438-54105ff71bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:11.639506 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b9381-c688-47f8-8438-54105ff71bc8 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:11.640348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b9381-c688-47f8-8438-54105ff71bc8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:11.640656 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b9381-c688-47f8-8438-54105ff71bc8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:11.642276 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b9381-c688-47f8-8438-54105ff71bc8 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:11.664739 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 509/1049] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 21:19:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:11.668835 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b9381-c688-47f8-8438-54105ff71bc8 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:11.710960 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125bee1b-d24c-43eb-856a-868eb05a4b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:11.712102 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125bee1b-d24c-43eb-856a-868eb05a4b2a None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:11.712582 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125bee1b-d24c-43eb-856a-868eb05a4b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:11.713071 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125bee1b-d24c-43eb-856a-868eb05a4b2a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:11.714629 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125bee1b-d24c-43eb-856a-868eb05a4b2a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:11.717226 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125bee1b-d24c-43eb-856a-868eb05a4b2a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:11.755124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 541/1050] 10.222.0.65 () {60 vars in 1144 bytes} [Thu May 19 21:19:11 2022] POST /identity/v3/endpoints => generated 327 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:19:11.821934 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:11.877849 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b9381-c688-47f8-8438-54105ff71bc8 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:11.967029 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d820f-d7d1-4dfd-b500-0db7f9b468ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:11.968732 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d820f-d7d1-4dfd-b500-0db7f9b468ee None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:11.969560 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d820f-d7d1-4dfd-b500-0db7f9b468ee None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:11.970416 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d820f-d7d1-4dfd-b500-0db7f9b468ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:11.972538 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6d820f-d7d1-4dfd-b500-0db7f9b468ee None admin] RBAC: Adding query filter params (role.id=bdd2ca99a0d74237b4642eae68a9a1c5, scope.project.id=c4940befa9c4405699c72deedc8a2127, user.id=e254671b21fe4661a381b9521810cf80) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:11.973581 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6d820f-d7d1-4dfd-b500-0db7f9b468ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:11.976779 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6d820f-d7d1-4dfd-b500-0db7f9b468ee None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:11.994281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 510/1051] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:19:11 2022] GET /identity/v3/role_assignments?user.id=e254671b21fe4661a381b9521810cf80&scope.project.id=c4940befa9c4405699c72deedc8a2127&role.id=bdd2ca99a0d74237b4642eae68a9a1c5 => generated 259 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:19:14.100202 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8702f-2012-4df7-8a9d-4e5dd2f4e486 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:14.101072 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8702f-2012-4df7-8a9d-4e5dd2f4e486 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:14.101072 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8702f-2012-4df7-8a9d-4e5dd2f4e486 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:14.102397 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 542/1052] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:19:14.107988 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6984ea03-fe9d-43eb-95c3-8896eeabb6a7 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:14.108737 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6984ea03-fe9d-43eb-95c3-8896eeabb6a7 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:14.108737 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6984ea03-fe9d-43eb-95c3-8896eeabb6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:14.126515 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-6984ea03-fe9d-43eb-95c3-8896eeabb6a7 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:19:14.127707 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6984ea03-fe9d-43eb-95c3-8896eeabb6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:19:14.213046 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 511/1053] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:19:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:19:14.238978 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e8702f-2012-4df7-8a9d-4e5dd2f4e486 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:14.286497 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:14.364708 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e8702f-2012-4df7-8a9d-4e5dd2f4e486 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:14.405493 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cce38e-e8d2-4067-9394-4bea45ac4378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:14.406374 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cce38e-e8d2-4067-9394-4bea45ac4378 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:14.406661 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cce38e-e8d2-4067-9394-4bea45ac4378 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:14.406936 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cce38e-e8d2-4067-9394-4bea45ac4378 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:14.422081 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cce38e-e8d2-4067-9394-4bea45ac4378 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:19:14.430149 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-d4cce38e-e8d2-4067-9394-4bea45ac4378 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:19:14.430149 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:19:14.430149 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:19:14.430149 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:19:14.430149 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:19:14.430149 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:19:14.430149 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:19:14.430149 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:19:14.430149 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:19:14.430149 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:19:14.430149 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:19:14.430149 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:19:14.430149 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:19:14.430149 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:19:14.430149 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:19:14.430149 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:19:14.432143 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:19:14.432143 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:19:14.432143 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:19:14.432143 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:19:14.432143 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:19:14.432143 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:19:14.432143 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:19:14.432143 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:19:14.432143 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:19:14.432143 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:19:14.432143 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:19:14.432143 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:19:14.432143 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:19:14.432143 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:19:14.432143 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:19:14.432143 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:19:14.433862 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:19:14.433862 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:19:14.433862 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:19:14.433862 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:19:14.433862 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:19:14.433862 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:19:14.433862 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:19:14.433862 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:19:14.433862 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:19:14.433862 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:19:14.433862 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:19:14.433862 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:19:14.443392 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 543/1054] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 21:19:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:19:14.443612 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6984ea03-fe9d-43eb-95c3-8896eeabb6a7 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:14.470399 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe57472-0e9c-4787-b890-f6042fb4839b None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:14.470911 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe57472-0e9c-4787-b890-f6042fb4839b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:14.471403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe57472-0e9c-4787-b890-f6042fb4839b None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:14.474660 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 544/1055] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:19:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:19:14.483359 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88896e-273e-4d31-9f16-a473b994bdd2 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:14.483859 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88896e-273e-4d31-9f16-a473b994bdd2 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:14.484366 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88896e-273e-4d31-9f16-a473b994bdd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:14.515030 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-7c88896e-273e-4d31-9f16-a473b994bdd2 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:19:14.517163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7c88896e-273e-4d31-9f16-a473b994bdd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:19:14.536852 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:14.600228 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6984ea03-fe9d-43eb-95c3-8896eeabb6a7 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:14.637720 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 545/1056] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:19:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:19:14.659292 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd977b-cc36-4295-b58b-229575d3a4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:14.660206 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd977b-cc36-4295-b58b-229575d3a4c5 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:14.660622 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd977b-cc36-4295-b58b-229575d3a4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:14.661060 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd977b-cc36-4295-b58b-229575d3a4c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:14.662032 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd977b-cc36-4295-b58b-229575d3a4c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:14.662488 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd977b-cc36-4295-b58b-229575d3a4c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:14.664047 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd977b-cc36-4295-b58b-229575d3a4c5 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:14.678490 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 512/1057] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:19:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:14.680313 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c88896e-273e-4d31-9f16-a473b994bdd2 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:14.681314 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd977b-cc36-4295-b58b-229575d3a4c5 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:14.715691 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:14.822605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c88896e-273e-4d31-9f16-a473b994bdd2 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:14.864038 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd977b-cc36-4295-b58b-229575d3a4c5 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:14.877262 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1223345-7515-4654-a2bd-63fa8fd7376c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:14.878668 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1223345-7515-4654-a2bd-63fa8fd7376c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:14.879459 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1223345-7515-4654-a2bd-63fa8fd7376c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:14.880006 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1223345-7515-4654-a2bd-63fa8fd7376c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:14.894148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1223345-7515-4654-a2bd-63fa8fd7376c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:19:14.903776 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-f1223345-7515-4654-a2bd-63fa8fd7376c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 21:19:14.903776 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:19:14.903776 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:19:14.903776 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:19:14.903776 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:19:14.903776 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:19:14.903776 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:19:14.903776 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:19:14.903776 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:19:14.903776 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:19:14.903776 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:19:14.903776 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:19:14.903776 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:19:14.903776 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:19:14.903776 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:19:14.903776 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:19:14.905587 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:19:14.905587 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:19:14.905587 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:19:14.905587 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:19:14.905587 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:19:14.905587 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:19:14.905587 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:19:14.905587 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:19:14.905587 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:19:14.905587 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:19:14.905587 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:19:14.905587 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:19:14.905587 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:19:14.905587 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:19:14.905587 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:19:14.905587 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:19:14.906699 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:19:14.906699 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:19:14.906699 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:19:14.906699 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:19:14.906699 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:19:14.906699 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:19:14.906699 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:19:14.906699 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:19:14.906699 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:19:14.906699 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:19:14.906699 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 21:19:14.906699 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:19:14.906699 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 546/1058] 10.222.0.65 () {56 vars in 1120 bytes} [Thu May 19 21:19:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:19:14.915667 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1223345-7515-4654-a2bd-63fa8fd7376c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:14.919038 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:14.930534 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69acdf-4aa6-4549-8e1b-e7007cef63dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:14.932258 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69acdf-4aa6-4549-8e1b-e7007cef63dd None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:14.932880 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69acdf-4aa6-4549-8e1b-e7007cef63dd None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:14.933357 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69acdf-4aa6-4549-8e1b-e7007cef63dd None admin] PATH_INFO: `/v3/users/dfebd80c5e974f8b849fa63e8ef18f42` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:14.958854 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b69acdf-4aa6-4549-8e1b-e7007cef63dd None admin] RBAC: Authorizing `identity:get_user(user_id=dfebd80c5e974f8b849fa63e8ef18f42)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:14.961543 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b69acdf-4aa6-4549-8e1b-e7007cef63dd None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:14.965108 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 513/1059] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:19:14 2022] GET /identity/v3/users/dfebd80c5e974f8b849fa63e8ef18f42 => generated 285 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:14.975450 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69acdf-4aa6-4549-8e1b-e7007cef63dd None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:15.052289 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:15.086029 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1223345-7515-4654-a2bd-63fa8fd7376c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:15.139844 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69acdf-4aa6-4549-8e1b-e7007cef63dd None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:15.142696 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039bf01e-9729-4c87-b190-c4e573a97c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:15.143815 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039bf01e-9729-4c87-b190-c4e573a97c0d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:15.144355 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039bf01e-9729-4c87-b190-c4e573a97c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:15.144836 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039bf01e-9729-4c87-b190-c4e573a97c0d None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:15.146574 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039bf01e-9729-4c87-b190-c4e573a97c0d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:15.147116 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039bf01e-9729-4c87-b190-c4e573a97c0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:15.149773 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039bf01e-9729-4c87-b190-c4e573a97c0d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:15.175246 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 547/1060] 10.222.0.65 () {56 vars in 1115 bytes} [Thu May 19 21:19:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:19:15.175246 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4861f-a6a4-426e-b020-33d47c9d2765 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:15.175246 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4861f-a6a4-426e-b020-33d47c9d2765 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:15.175246 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4861f-a6a4-426e-b020-33d47c9d2765 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:15.176812 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 548/1061] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:19:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:19:15.177684 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a4861f-a6a4-426e-b020-33d47c9d2765 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:15.198269 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f9deca-6d45-423a-ac31-b1c82ad9abc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:15.199191 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9deca-6d45-423a-ac31-b1c82ad9abc6 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:15.199639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9deca-6d45-423a-ac31-b1c82ad9abc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:15.200114 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9deca-6d45-423a-ac31-b1c82ad9abc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:15.283079 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:15.300480 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f9deca-6d45-423a-ac31-b1c82ad9abc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:15.302304 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f9deca-6d45-423a-ac31-b1c82ad9abc6 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:15.316754 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a4861f-a6a4-426e-b020-33d47c9d2765 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:15.368622 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc36815-1594-4247-bdf6-5bbfdfa0f96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:15.369959 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc36815-1594-4247-bdf6-5bbfdfa0f96d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:15.370534 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc36815-1594-4247-bdf6-5bbfdfa0f96d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:15.371009 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc36815-1594-4247-bdf6-5bbfdfa0f96d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:15.391360 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc36815-1594-4247-bdf6-5bbfdfa0f96d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:15.393102 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc36815-1594-4247-bdf6-5bbfdfa0f96d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:15.403638 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:15.416720 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 514/1062] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:19:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 21:19:15.417176 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-5bc36815-1594-4247-bdf6-5bbfdfa0f96d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:19:15.417176 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:19:15.417176 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:19:15.417176 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:19:15.417176 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:19:15.417176 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:19:15.417176 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:19:15.417176 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:19:15.417176 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:19:15.417176 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:19:15.417176 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:19:15.417176 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:19:15.417176 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:19:15.417176 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:19:15.417176 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:19:15.418863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:19:15.418863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:19:15.418863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:19:15.418863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:19:15.418863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:19:15.418863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:19:15.418863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:19:15.418863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:19:15.418863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:19:15.418863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:19:15.418863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:19:15.418863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:19:15.418863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:19:15.418863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:19:15.418863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:19:15.418863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:19:15.418863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:19:15.420613 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:19:15.420613 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:19:15.420613 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:19:15.420613 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:19:15.421030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009708d5-3504-4647-981d-07d76a7862cb None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:15.421030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009708d5-3504-4647-981d-07d76a7862cb None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:15.421030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009708d5-3504-4647-981d-07d76a7862cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:15.421390 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 549/1063] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:19:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:19:15.425168 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc36815-1594-4247-bdf6-5bbfdfa0f96d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:15.448464 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-009708d5-3504-4647-981d-07d76a7862cb None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:19:15.450430 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-009708d5-3504-4647-981d-07d76a7862cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:19:15.547503 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc36815-1594-4247-bdf6-5bbfdfa0f96d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:15.552640 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 515/1064] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:19:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:19:15.554114 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009708d5-3504-4647-981d-07d76a7862cb None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:15.586904 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:15.590452 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a3c91-d329-4875-9f86-4fde818ca027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:15.591835 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a3c91-d329-4875-9f86-4fde818ca027 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:15.593036 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a3c91-d329-4875-9f86-4fde818ca027 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:15.593523 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a3c91-d329-4875-9f86-4fde818ca027 None admin] PATH_INFO: `/v3/projects/1e139d99242f43eaa802c7e7be02bdb8` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:15.595800 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674a3c91-d329-4875-9f86-4fde818ca027 None admin] RBAC: Authorizing `identity:get_project(project_id=1e139d99242f43eaa802c7e7be02bdb8)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:15.599292 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674a3c91-d329-4875-9f86-4fde818ca027 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:15.602639 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 550/1065] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:19:15 2022] GET /identity/v3/projects/1e139d99242f43eaa802c7e7be02bdb8 => generated 311 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:15.603722 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a3c91-d329-4875-9f86-4fde818ca027 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:15.712006 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009708d5-3504-4647-981d-07d76a7862cb None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:15.713941 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:15.755693 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a3c91-d329-4875-9f86-4fde818ca027 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:15.756182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae0b277-c2bd-4708-80f0-3f8a5770477f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:15.756974 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae0b277-c2bd-4708-80f0-3f8a5770477f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:15.757340 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae0b277-c2bd-4708-80f0-3f8a5770477f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:15.757706 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae0b277-c2bd-4708-80f0-3f8a5770477f None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:15.758713 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae0b277-c2bd-4708-80f0-3f8a5770477f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:15.759118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae0b277-c2bd-4708-80f0-3f8a5770477f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:15.761098 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae0b277-c2bd-4708-80f0-3f8a5770477f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:15.777028 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 516/1066] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:19:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:15.778056 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae0b277-c2bd-4708-80f0-3f8a5770477f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:15.805216 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3fca99-5967-47d1-b6ea-b781381a9a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:15.806310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3fca99-5967-47d1-b6ea-b781381a9a82 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:15.806763 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3fca99-5967-47d1-b6ea-b781381a9a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:15.807206 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3fca99-5967-47d1-b6ea-b781381a9a82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:15.890472 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:15.906450 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3fca99-5967-47d1-b6ea-b781381a9a82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:15.908047 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3fca99-5967-47d1-b6ea-b781381a9a82 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:15.919823 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae0b277-c2bd-4708-80f0-3f8a5770477f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:15.970368 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a2306-0148-4881-8030-0eef228c1525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:15.971306 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a2306-0148-4881-8030-0eef228c1525 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:15.971750 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a2306-0148-4881-8030-0eef228c1525 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:15.972082 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a2306-0148-4881-8030-0eef228c1525 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:15.973148 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85a2306-0148-4881-8030-0eef228c1525 None admin] RBAC: Adding query filter params (role.id=2b714f2e5c6c469f8fa037bda1ed16f6, scope.project.id=1e139d99242f43eaa802c7e7be02bdb8, user.id=dfebd80c5e974f8b849fa63e8ef18f42) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:15.973590 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85a2306-0148-4881-8030-0eef228c1525 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:15.975393 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85a2306-0148-4881-8030-0eef228c1525 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:15.998279 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 517/1067] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:19:15 2022] GET /identity/v3/role_assignments?user.id=dfebd80c5e974f8b849fa63e8ef18f42&scope.project.id=1e139d99242f43eaa802c7e7be02bdb8&role.id=2b714f2e5c6c469f8fa037bda1ed16f6 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:15.998279 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a2306-0148-4881-8030-0eef228c1525 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:16.015065 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:16.063410 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 551/1068] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:19:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 21:19:16.070170 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3fca99-5967-47d1-b6ea-b781381a9a82 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:16.180398 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a2306-0148-4881-8030-0eef228c1525 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:16.215347 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3fca99-5967-47d1-b6ea-b781381a9a82 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:16.237809 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5342c9-a46e-439d-9b12-8a6668e60549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:16.239560 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5342c9-a46e-439d-9b12-8a6668e60549 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:16.240026 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5342c9-a46e-439d-9b12-8a6668e60549 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:16.240493 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5342c9-a46e-439d-9b12-8a6668e60549 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:16.255221 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5342c9-a46e-439d-9b12-8a6668e60549 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:16.256127 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ef5e0-a8d5-4c70-91dc-f38c91dbb4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:16.257110 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ef5e0-a8d5-4c70-91dc-f38c91dbb4f8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:16.257540 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ef5e0-a8d5-4c70-91dc-f38c91dbb4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:16.257959 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ef5e0-a8d5-4c70-91dc-f38c91dbb4f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:16.258147 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5342c9-a46e-439d-9b12-8a6668e60549 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:16.270657 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ef5e0-a8d5-4c70-91dc-f38c91dbb4f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:16.272441 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ef5e0-a8d5-4c70-91dc-f38c91dbb4f8 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:16.287913 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-dc5342c9-a46e-439d-9b12-8a6668e60549 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 21:19:16.287913 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:19:16.287913 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:19:16.287913 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:19:16.287913 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:19:16.287913 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:19:16.287913 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:19:16.287913 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:19:16.287913 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:19:16.287913 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:19:16.287913 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:19:16.287913 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:19:16.287913 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:19:16.287913 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:19:16.287913 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:19:16.287913 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:19:16.289088 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:19:16.289088 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:19:16.289088 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:19:16.289088 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:19:16.289088 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:19:16.289088 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:19:16.289088 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:19:16.289088 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:19:16.289088 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:19:16.289088 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:19:16.289088 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:19:16.289088 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:19:16.289088 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:19:16.289088 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:19:16.289088 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:19:16.289088 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:19:16.289088 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:19:16.289088 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:19:16.290447 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:19:16.290447 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:19:16.290447 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:19:16.290447 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:19:16.290447 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:19:16.290447 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:19:16.290447 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:19:16.290447 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:19:16.290447 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 21:19:16.290447 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:19:16.290447 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 518/1069] 10.222.0.65 () {56 vars in 1093 bytes} [Thu May 19 21:19:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:19:16.295656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-702ef5e0-a8d5-4c70-91dc-f38c91dbb4f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:19:16.295656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:19:16.295656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:19:16.295656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:19:16.295656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:19:16.295656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:19:16.295656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:19:16.295656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:19:16.295656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:19:16.295656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:19:16.295656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:19:16.295656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:19:16.295656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:19:16.295656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:19:16.295656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:19:16.298822 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:19:16.298822 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:19:16.298822 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:19:16.298822 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:19:16.298822 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:19:16.298822 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:19:16.298822 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:19:16.298822 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:19:16.298822 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:19:16.298822 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:19:16.298822 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:19:16.298822 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:19:16.298822 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:19:16.298822 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:19:16.298822 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:19:16.298822 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:19:16.298822 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:19:16.301297 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:19:16.301297 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:19:16.301297 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:19:16.301297 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:19:16.301297 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 552/1070] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:19:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:19:16.301826 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5342c9-a46e-439d-9b12-8a6668e60549 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:16.303476 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ef5e0-a8d5-4c70-91dc-f38c91dbb4f8 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:16.306570 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:16.339243 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:16.438500 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5342c9-a46e-439d-9b12-8a6668e60549 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:16.466444 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ef5e0-a8d5-4c70-91dc-f38c91dbb4f8 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:16.477902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a706b88-6c64-44ed-ac07-b59fb0be7b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:16.478831 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a706b88-6c64-44ed-ac07-b59fb0be7b12 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:16.479240 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a706b88-6c64-44ed-ac07-b59fb0be7b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:16.479623 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a706b88-6c64-44ed-ac07-b59fb0be7b12 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:16.480474 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a706b88-6c64-44ed-ac07-b59fb0be7b12 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:16.480776 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a706b88-6c64-44ed-ac07-b59fb0be7b12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:16.482304 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a706b88-6c64-44ed-ac07-b59fb0be7b12 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:16.508644 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 519/1071] 10.222.0.65 () {56 vars in 1133 bytes} [Thu May 19 21:19:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:16.514124 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a706b88-6c64-44ed-ac07-b59fb0be7b12 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:16.527904 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01fc00e-9975-49c3-8945-6e511b73cb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:16.528954 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01fc00e-9975-49c3-8945-6e511b73cb6b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:16.529427 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01fc00e-9975-49c3-8945-6e511b73cb6b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:16.529896 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01fc00e-9975-49c3-8945-6e511b73cb6b None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:16.531192 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01fc00e-9975-49c3-8945-6e511b73cb6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:16.531692 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01fc00e-9975-49c3-8945-6e511b73cb6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:16.534088 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01fc00e-9975-49c3-8945-6e511b73cb6b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:16.555414 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 553/1072] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:19:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:16.621744 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:16.693767 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a706b88-6c64-44ed-ac07-b59fb0be7b12 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:16.755965 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d72367b-e4bb-4ae5-acb3-becd1439c0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:16.757258 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d72367b-e4bb-4ae5-acb3-becd1439c0d7 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:16.757683 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d72367b-e4bb-4ae5-acb3-becd1439c0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:16.758095 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d72367b-e4bb-4ae5-acb3-becd1439c0d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:16.777493 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d72367b-e4bb-4ae5-acb3-becd1439c0d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:16.780219 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d72367b-e4bb-4ae5-acb3-becd1439c0d7 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:16.803794 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-7d72367b-e4bb-4ae5-acb3-becd1439c0d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:19:16.803794 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:19:16.803794 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:19:16.803794 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:19:16.803794 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:19:16.803794 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:19:16.803794 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:19:16.803794 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:19:16.803794 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:19:16.803794 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:19:16.803794 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:19:16.803794 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:19:16.803794 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:19:16.803794 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:19:16.803794 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:19:16.811183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:19:16.811183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:19:16.811183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:19:16.811183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:19:16.811183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:19:16.811183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:19:16.811183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:19:16.811183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:19:16.811183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:19:16.811183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:19:16.811183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:19:16.811183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:19:16.811183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:19:16.811183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:19:16.811183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:19:16.811183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:19:16.811183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:19:16.812975 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:19:16.812975 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:19:16.812975 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:19:16.812975 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:19:16.812975 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 520/1073] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:19:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:19:16.818988 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01fc00e-9975-49c3-8945-6e511b73cb6b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:16.938694 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:16.967664 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01fc00e-9975-49c3-8945-6e511b73cb6b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:17.011076 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9207d208-7ad6-4dd4-badd-1b691d83c8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:17.011893 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207d208-7ad6-4dd4-badd-1b691d83c8c4 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:17.012181 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207d208-7ad6-4dd4-badd-1b691d83c8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:17.012460 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207d208-7ad6-4dd4-badd-1b691d83c8c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:17.013231 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9207d208-7ad6-4dd4-badd-1b691d83c8c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:17.013541 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9207d208-7ad6-4dd4-badd-1b691d83c8c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:17.015039 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9207d208-7ad6-4dd4-badd-1b691d83c8c4 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:17.028913 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 554/1074] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:19:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:17.034556 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d72367b-e4bb-4ae5-acb3-becd1439c0d7 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:17.154843 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d72367b-e4bb-4ae5-acb3-becd1439c0d7 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:17.169483 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:17.194417 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1866fc0b-3bfa-471b-a184-df9bd9b16707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:17.195277 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1866fc0b-3bfa-471b-a184-df9bd9b16707 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:17.195552 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1866fc0b-3bfa-471b-a184-df9bd9b16707 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:17.195827 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1866fc0b-3bfa-471b-a184-df9bd9b16707 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:17.203983 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1866fc0b-3bfa-471b-a184-df9bd9b16707 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:17.205456 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1866fc0b-3bfa-471b-a184-df9bd9b16707 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:17.220491 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-1866fc0b-3bfa-471b-a184-df9bd9b16707 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:19:17.220491 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:19:17.220491 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:19:17.220491 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:19:17.220491 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:19:17.220491 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:19:17.220491 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:19:17.220491 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:19:17.220491 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:19:17.220491 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:19:17.220491 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:19:17.220491 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:19:17.220491 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:19:17.220491 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:19:17.220491 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:19:17.221704 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:19:17.221704 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:19:17.221704 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:19:17.221704 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:19:17.221704 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:19:17.221704 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:19:17.221704 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:19:17.221704 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:19:17.221704 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:19:17.221704 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:19:17.221704 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:19:17.221704 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:19:17.221704 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:19:17.221704 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:19:17.221704 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:19:17.221704 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:19:17.221704 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:19:17.222955 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:19:17.222955 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:19:17.222955 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:19:17.222955 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:19:17.222955 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:19:17.222955 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:19:17.222955 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:19:17.222955 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:19:17.222955 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 521/1075] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:19:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:19:17.227804 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9207d208-7ad6-4dd4-badd-1b691d83c8c4 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:17.333974 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9207d208-7ad6-4dd4-badd-1b691d83c8c4 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:17.372460 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f453e-f6f9-474a-9c16-5bd9ed797c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:17.373124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f453e-f6f9-474a-9c16-5bd9ed797c1e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:17.373406 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f453e-f6f9-474a-9c16-5bd9ed797c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:17.373668 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f453e-f6f9-474a-9c16-5bd9ed797c1e None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:17.374484 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994f453e-f6f9-474a-9c16-5bd9ed797c1e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:17.374785 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994f453e-f6f9-474a-9c16-5bd9ed797c1e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:17.376235 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994f453e-f6f9-474a-9c16-5bd9ed797c1e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:17.385723 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:17.391887 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 555/1076] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 21:19:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:17.397229 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1866fc0b-3bfa-471b-a184-df9bd9b16707 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:17.508957 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1866fc0b-3bfa-471b-a184-df9bd9b16707 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:17.552620 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13f0cb0-762c-4a50-9303-5c64f1c82adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:17.553576 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13f0cb0-762c-4a50-9303-5c64f1c82adc None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:17.553863 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13f0cb0-762c-4a50-9303-5c64f1c82adc None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:17.554138 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13f0cb0-762c-4a50-9303-5c64f1c82adc None admin] PATH_INFO: `/v3/projects/c4940befa9c4405699c72deedc8a2127/users/e254671b21fe4661a381b9521810cf80/roles/bdd2ca99a0d74237b4642eae68a9a1c5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:17.556570 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13f0cb0-762c-4a50-9303-5c64f1c82adc None admin] RBAC: Authorizing `identity:create_grant(project_id=c4940befa9c4405699c72deedc8a2127, user_id=e254671b21fe4661a381b9521810cf80, role_id=bdd2ca99a0d74237b4642eae68a9a1c5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:17.558025 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13f0cb0-762c-4a50-9303-5c64f1c82adc None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:17.573555 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 522/1077] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 21:19:17 2022] PUT /identity/v3/projects/c4940befa9c4405699c72deedc8a2127/users/e254671b21fe4661a381b9521810cf80/roles/bdd2ca99a0d74237b4642eae68a9a1c5 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:19:18.322892 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed3441-c78e-4573-bace-f695c3edf637 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:18.323526 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed3441-c78e-4573-bace-f695c3edf637 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:18.323526 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed3441-c78e-4573-bace-f695c3edf637 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:18.325113 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 556/1078] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:19:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:19:18.330693 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a94af7-d55f-4880-8c2d-514c36c043a4 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:18.331639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a94af7-d55f-4880-8c2d-514c36c043a4 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:18.331639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a94af7-d55f-4880-8c2d-514c36c043a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:18.350187 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-13a94af7-d55f-4880-8c2d-514c36c043a4 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:19:18.351460 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-13a94af7-d55f-4880-8c2d-514c36c043a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:19:18.385465 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:18.432149 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 523/1079] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:19:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:19:18.454729 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed3441-c78e-4573-bace-f695c3edf637 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:18.583748 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed3441-c78e-4573-bace-f695c3edf637 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:18.623954 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c13040-b874-4668-aa63-8b62a02dc99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:18.624782 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c13040-b874-4668-aa63-8b62a02dc99e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:18.625059 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c13040-b874-4668-aa63-8b62a02dc99e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:18.625332 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c13040-b874-4668-aa63-8b62a02dc99e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:18.646118 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c13040-b874-4668-aa63-8b62a02dc99e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:19:18.654163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-d5c13040-b874-4668-aa63-8b62a02dc99e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:19:18.654163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:19:18.654163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:19:18.654163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:19:18.654163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:19:18.654163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:19:18.654163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:19:18.654163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:19:18.654163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:19:18.654163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:19:18.654163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:19:18.654163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:19:18.654163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:19:18.654163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:19:18.654163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:19:18.654163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:19:18.655895 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:19:18.655895 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:19:18.655895 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:19:18.655895 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:19:18.655895 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:19:18.655895 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:19:18.655895 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:19:18.655895 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:19:18.655895 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:19:18.655895 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:19:18.655895 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:19:18.655895 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:19:18.655895 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:19:18.655895 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:19:18.655895 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:19:18.655895 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:19:18.657573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:19:18.657573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:19:18.657573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:19:18.657573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:19:18.657573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:19:18.657573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:19:18.657573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:19:18.657573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:19:18.657573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:19:18.657573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:19:18.657573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:19:18.657573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:19:18.657573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 557/1080] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 21:19:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:19:18.662766 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a94af7-d55f-4880-8c2d-514c36c043a4 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:18.812580 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a94af7-d55f-4880-8c2d-514c36c043a4 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:18.852654 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3588a8ba-6496-42a7-b8cb-a7945a99f6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:18.853356 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588a8ba-6496-42a7-b8cb-a7945a99f6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:18.853632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588a8ba-6496-42a7-b8cb-a7945a99f6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:18.853901 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588a8ba-6496-42a7-b8cb-a7945a99f6c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:18.854710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3588a8ba-6496-42a7-b8cb-a7945a99f6c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:18.854999 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3588a8ba-6496-42a7-b8cb-a7945a99f6c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:18.856634 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3588a8ba-6496-42a7-b8cb-a7945a99f6c0 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:18.867035 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 524/1081] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:19:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:19:18.873206 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c13040-b874-4668-aa63-8b62a02dc99e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:18.943789 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:18.998028 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c13040-b874-4668-aa63-8b62a02dc99e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:19.044258 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e17139-cd51-44f4-98db-64fbb0ab7f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:19.045290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e17139-cd51-44f4-98db-64fbb0ab7f0f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:19.045629 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e17139-cd51-44f4-98db-64fbb0ab7f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:19.045965 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e17139-cd51-44f4-98db-64fbb0ab7f0f None admin] PATH_INFO: `/v3/users/dfebd80c5e974f8b849fa63e8ef18f42` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:19.047502 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e17139-cd51-44f4-98db-64fbb0ab7f0f None admin] RBAC: Authorizing `identity:get_user(user_id=dfebd80c5e974f8b849fa63e8ef18f42)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:19.049089 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e17139-cd51-44f4-98db-64fbb0ab7f0f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:19.051742 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 558/1082] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:19:18 2022] GET /identity/v3/users/dfebd80c5e974f8b849fa63e8ef18f42 => generated 285 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:19.057941 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3588a8ba-6496-42a7-b8cb-a7945a99f6c0 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:19.091476 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:19.190357 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3588a8ba-6496-42a7-b8cb-a7945a99f6c0 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:19.231276 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76094f-7633-4f0f-a01d-1bae5695a7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:19.232192 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76094f-7633-4f0f-a01d-1bae5695a7b7 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:19.232480 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76094f-7633-4f0f-a01d-1bae5695a7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:19.232739 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76094f-7633-4f0f-a01d-1bae5695a7b7 None admin] PATH_INFO: `/v3/projects/1e139d99242f43eaa802c7e7be02bdb8` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:19.234034 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b76094f-7633-4f0f-a01d-1bae5695a7b7 None admin] RBAC: Authorizing `identity:get_project(project_id=1e139d99242f43eaa802c7e7be02bdb8)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:19.235569 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b76094f-7633-4f0f-a01d-1bae5695a7b7 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:19.237333 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 525/1083] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:19:19 2022] GET /identity/v3/projects/1e139d99242f43eaa802c7e7be02bdb8 => generated 311 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:19.247615 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e17139-cd51-44f4-98db-64fbb0ab7f0f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:19.369278 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e17139-cd51-44f4-98db-64fbb0ab7f0f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:19.402920 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:19.413400 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a531fca-83fc-4fd4-a34e-34f5d330ade8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:19.414625 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a531fca-83fc-4fd4-a34e-34f5d330ade8 None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:19.415061 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a531fca-83fc-4fd4-a34e-34f5d330ade8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:19.415425 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a531fca-83fc-4fd4-a34e-34f5d330ade8 None admin] PATH_INFO: `/v3/projects/1e139d99242f43eaa802c7e7be02bdb8/users/dfebd80c5e974f8b849fa63e8ef18f42/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:19.417638 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a531fca-83fc-4fd4-a34e-34f5d330ade8 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e139d99242f43eaa802c7e7be02bdb8, user_id=dfebd80c5e974f8b849fa63e8ef18f42, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:19.419249 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a531fca-83fc-4fd4-a34e-34f5d330ade8 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:19.435821 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 559/1084] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 21:19:19 2022] PUT /identity/v3/projects/1e139d99242f43eaa802c7e7be02bdb8/users/dfebd80c5e974f8b849fa63e8ef18f42/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:19:19.847672 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4647c6c-fcf4-4ee4-8637-79c52695ba78 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:19.848247 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4647c6c-fcf4-4ee4-8637-79c52695ba78 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:19.848692 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4647c6c-fcf4-4ee4-8637-79c52695ba78 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:19.851167 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 526/1085] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:19:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:19:19.859407 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d72c52-7588-491b-b516-74581f808d94 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:19.859902 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d72c52-7588-491b-b516-74581f808d94 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:19.860469 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d72c52-7588-491b-b516-74581f808d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:19.881427 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-60d72c52-7588-491b-b516-74581f808d94 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:19:19.882734 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-60d72c52-7588-491b-b516-74581f808d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:19:19.966063 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 560/1086] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:19:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:19:19.991408 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4647c6c-fcf4-4ee4-8637-79c52695ba78 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:20.098072 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:20.116427 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4647c6c-fcf4-4ee4-8637-79c52695ba78 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:20.178598 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6d82f-6849-4de5-80e2-73da4cb38901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:20.180071 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6d82f-6849-4de5-80e2-73da4cb38901 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:20.180557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6d82f-6849-4de5-80e2-73da4cb38901 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:20.181053 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6d82f-6849-4de5-80e2-73da4cb38901 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:20.202151 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a6d82f-6849-4de5-80e2-73da4cb38901 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:19:20.219057 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-d2a6d82f-6849-4de5-80e2-73da4cb38901 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 21:19:20.219057 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:19:20.219057 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:19:20.219057 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:19:20.219057 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:19:20.219057 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:19:20.219057 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:19:20.219057 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:19:20.219057 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:19:20.219057 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:19:20.219057 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:19:20.219057 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:19:20.219057 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:19:20.219057 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:19:20.219057 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:19:20.219057 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:19:20.220255 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:19:20.220255 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:19:20.220255 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:19:20.220255 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:19:20.220255 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:19:20.220255 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:19:20.220255 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:19:20.220255 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:19:20.220255 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:19:20.220255 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:19:20.220255 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:19:20.220255 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:19:20.220255 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:19:20.220255 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:19:20.220255 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:19:20.220255 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:19:20.221102 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:19:20.221102 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:19:20.221102 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:19:20.221102 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:19:20.221102 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:19:20.221102 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:19:20.221102 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:19:20.221102 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:19:20.221102 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:19:20.221102 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:19:20.221102 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 21:19:20.221102 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:19:20.221995 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 527/1087] 10.222.0.65 () {56 vars in 1120 bytes} [Thu May 19 21:19:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:19:20.229891 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d72c52-7588-491b-b516-74581f808d94 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:20.349508 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d72c52-7588-491b-b516-74581f808d94 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:20.371316 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:20.392398 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5925baf-d4ef-4915-a4d4-7a05a7eba6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:20.393091 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5925baf-d4ef-4915-a4d4-7a05a7eba6c9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:20.393373 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5925baf-d4ef-4915-a4d4-7a05a7eba6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:20.393642 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5925baf-d4ef-4915-a4d4-7a05a7eba6c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:20.394508 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5925baf-d4ef-4915-a4d4-7a05a7eba6c9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:20.394999 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5925baf-d4ef-4915-a4d4-7a05a7eba6c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:20.396628 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5925baf-d4ef-4915-a4d4-7a05a7eba6c9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:20.407346 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 561/1088] 10.222.0.65 () {56 vars in 1115 bytes} [Thu May 19 21:19:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:20.414833 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6d82f-6849-4de5-80e2-73da4cb38901 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:20.619449 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6d82f-6849-4de5-80e2-73da4cb38901 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:20.680885 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70292563-384a-408c-9d5a-b5b4e1002f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:20.682227 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70292563-384a-408c-9d5a-b5b4e1002f18 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:20.682753 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70292563-384a-408c-9d5a-b5b4e1002f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:20.683251 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70292563-384a-408c-9d5a-b5b4e1002f18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:20.702512 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70292563-384a-408c-9d5a-b5b4e1002f18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:20.705426 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70292563-384a-408c-9d5a-b5b4e1002f18 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:20.742520 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-70292563-384a-408c-9d5a-b5b4e1002f18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:19:20.742520 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:19:20.742520 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:19:20.742520 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:19:20.742520 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:19:20.742520 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:19:20.742520 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:19:20.742520 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:19:20.742520 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:19:20.742520 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:19:20.742520 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:19:20.742520 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:19:20.742520 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:19:20.742520 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:19:20.742520 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:19:20.744379 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:19:20.744379 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:19:20.744379 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:19:20.744379 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:19:20.744379 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:19:20.744379 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:19:20.744379 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:19:20.744379 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:19:20.744379 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:19:20.744379 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:19:20.744379 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:19:20.744379 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:19:20.744379 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:19:20.744379 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:19:20.744379 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:19:20.744379 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:19:20.744379 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:19:20.751427 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:19:20.751427 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:19:20.751427 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:19:20.751427 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:19:20.751427 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 528/1089] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:19:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:19:20.753707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5925baf-d4ef-4915-a4d4-7a05a7eba6c9 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:20.892204 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5925baf-d4ef-4915-a4d4-7a05a7eba6c9 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:20.936740 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3db4de-5be5-4b4a-ad38-7f8132c1468c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:20.937544 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3db4de-5be5-4b4a-ad38-7f8132c1468c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:20.937913 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3db4de-5be5-4b4a-ad38-7f8132c1468c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:20.938342 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3db4de-5be5-4b4a-ad38-7f8132c1468c None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:20.939175 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3db4de-5be5-4b4a-ad38-7f8132c1468c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:20.939569 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3db4de-5be5-4b4a-ad38-7f8132c1468c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:20.941230 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3db4de-5be5-4b4a-ad38-7f8132c1468c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:20.956390 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 562/1090] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:19:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:20.963353 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70292563-384a-408c-9d5a-b5b4e1002f18 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:20.967650 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:21.083462 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70292563-384a-408c-9d5a-b5b4e1002f18 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:21.124761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f5985-1324-419e-bd99-8fa85bbd3823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:21.125580 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f5985-1324-419e-bd99-8fa85bbd3823 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:21.125860 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f5985-1324-419e-bd99-8fa85bbd3823 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:21.126136 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f5985-1324-419e-bd99-8fa85bbd3823 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:21.135177 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9f5985-1324-419e-bd99-8fa85bbd3823 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:21.136666 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9f5985-1324-419e-bd99-8fa85bbd3823 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:21.152992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-ce9f5985-1324-419e-bd99-8fa85bbd3823 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 21:19:21.152992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:19:21.152992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:19:21.152992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:19:21.152992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:19:21.152992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:19:21.152992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:19:21.152992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:19:21.152992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:19:21.152992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:19:21.152992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:19:21.152992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:19:21.152992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:19:21.152992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:19:21.152992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:19:21.152992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:19:21.155274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:19:21.155274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:19:21.155274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:19:21.155274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:19:21.155274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:19:21.155274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:19:21.155274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:19:21.155274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:19:21.155274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:19:21.155274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:19:21.155274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:19:21.155274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:19:21.155274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:19:21.155274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:19:21.155274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:19:21.155274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:19:21.155274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:19:21.155274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:19:21.157466 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:19:21.157466 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:19:21.157466 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:19:21.157466 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:19:21.157466 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:19:21.157466 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:19:21.157466 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:19:21.157466 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:19:21.157466 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 21:19:21.157466 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:19:21.159009 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 529/1091] 10.222.0.65 () {56 vars in 1093 bytes} [Thu May 19 21:19:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:19:21.160601 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3db4de-5be5-4b4a-ad38-7f8132c1468c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:21.221703 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:21.342849 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3db4de-5be5-4b4a-ad38-7f8132c1468c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:21.396757 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2947f297-a7c3-4d84-b5ad-533a84a2433e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:21.397763 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947f297-a7c3-4d84-b5ad-533a84a2433e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:21.397934 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947f297-a7c3-4d84-b5ad-533a84a2433e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:21.398230 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947f297-a7c3-4d84-b5ad-533a84a2433e None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:21.399279 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2947f297-a7c3-4d84-b5ad-533a84a2433e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:21.399656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2947f297-a7c3-4d84-b5ad-533a84a2433e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:21.401352 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2947f297-a7c3-4d84-b5ad-533a84a2433e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:21.426997 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 563/1092] 10.222.0.65 () {56 vars in 1133 bytes} [Thu May 19 21:19:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:21.433355 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f5985-1324-419e-bd99-8fa85bbd3823 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:21.516909 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:21.579541 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f5985-1324-419e-bd99-8fa85bbd3823 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:21.623049 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dd351b-1c92-4961-a604-d9623b61115a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:21.623713 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd351b-1c92-4961-a604-d9623b61115a None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:21.623989 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd351b-1c92-4961-a604-d9623b61115a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:21.624265 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd351b-1c92-4961-a604-d9623b61115a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:21.736034 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dd351b-1c92-4961-a604-d9623b61115a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:21.739562 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dd351b-1c92-4961-a604-d9623b61115a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:21.857262 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:21.933574 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 530/1093] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:19:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 21:19:21.939581 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2947f297-a7c3-4d84-b5ad-533a84a2433e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:21.963450 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61abeae-32fa-4775-8dc5-6af24c17742a None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:21.964271 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61abeae-32fa-4775-8dc5-6af24c17742a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:21.964271 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61abeae-32fa-4775-8dc5-6af24c17742a None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:21.966523 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 531/1094] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:19:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:19:21.972504 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5061e75-cb56-40f8-b442-04ab2245d6ad None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:21.972990 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5061e75-cb56-40f8-b442-04ab2245d6ad None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:21.973477 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5061e75-cb56-40f8-b442-04ab2245d6ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:21.995504 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-f5061e75-cb56-40f8-b442-04ab2245d6ad None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:19:21.997021 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f5061e75-cb56-40f8-b442-04ab2245d6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:19:22.073626 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:22.077946 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2947f297-a7c3-4d84-b5ad-533a84a2433e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:22.088211 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 532/1095] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:19:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:19:22.112029 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5061e75-cb56-40f8-b442-04ab2245d6ad None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:22.121390 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d906509d-98ae-4fb1-a3d4-3aca413a7c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:22.122338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906509d-98ae-4fb1-a3d4-3aca413a7c52 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:22.122779 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906509d-98ae-4fb1-a3d4-3aca413a7c52 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:22.123216 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906509d-98ae-4fb1-a3d4-3aca413a7c52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:22.141059 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d906509d-98ae-4fb1-a3d4-3aca413a7c52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:22.143339 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d906509d-98ae-4fb1-a3d4-3aca413a7c52 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:22.165688 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-d906509d-98ae-4fb1-a3d4-3aca413a7c52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:19:22.165688 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:19:22.165688 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:19:22.165688 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:19:22.165688 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:19:22.165688 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:19:22.165688 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:19:22.165688 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:19:22.165688 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:19:22.165688 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:19:22.165688 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:19:22.165688 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:19:22.165688 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:19:22.165688 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:19:22.165688 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:19:22.167474 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:19:22.167474 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:19:22.167474 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:19:22.167474 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:19:22.167474 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:19:22.167474 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:19:22.167474 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:19:22.167474 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:19:22.167474 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:19:22.167474 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:19:22.167474 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:19:22.167474 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:19:22.167474 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:19:22.167474 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:19:22.167474 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:19:22.167474 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:19:22.167474 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:19:22.169229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:19:22.169229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:19:22.169229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:19:22.169229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:19:22.169229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 564/1096] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:19:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:19:22.173714 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d906509d-98ae-4fb1-a3d4-3aca413a7c52 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:22.247165 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:22.256996 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5061e75-cb56-40f8-b442-04ab2245d6ad None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:22.305327 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754f76e-2b3a-4bf4-9750-caba0cc61a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:22.306230 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754f76e-2b3a-4bf4-9750-caba0cc61a24 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:22.306613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754f76e-2b3a-4bf4-9750-caba0cc61a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:22.307027 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754f76e-2b3a-4bf4-9750-caba0cc61a24 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:22.318879 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d906509d-98ae-4fb1-a3d4-3aca413a7c52 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:22.320776 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7754f76e-2b3a-4bf4-9750-caba0cc61a24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:19:22.329560 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-7754f76e-2b3a-4bf4-9750-caba0cc61a24 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:19:22.329560 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:19:22.329560 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:19:22.329560 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:19:22.329560 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:19:22.329560 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:19:22.329560 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:19:22.329560 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:19:22.329560 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:19:22.329560 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:19:22.329560 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:19:22.329560 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:19:22.329560 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:19:22.329560 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:19:22.329560 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:19:22.329560 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:19:22.332626 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:19:22.332626 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:19:22.332626 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:19:22.332626 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:19:22.332626 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:19:22.332626 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:19:22.332626 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:19:22.332626 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:19:22.332626 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:19:22.332626 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:19:22.332626 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:19:22.332626 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:19:22.332626 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:19:22.332626 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:19:22.332626 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:19:22.332626 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:19:22.334776 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:19:22.334776 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:19:22.334776 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:19:22.334776 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:19:22.334776 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:19:22.334776 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:19:22.334776 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:19:22.334776 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:19:22.334776 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:19:22.334776 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:19:22.334776 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:19:22.334776 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:19:22.334776 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 533/1097] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 21:19:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:19:22.339787 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754f76e-2b3a-4bf4-9750-caba0cc61a24 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:22.366659 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c7510-cf7b-4dc4-a021-82a0ecebea5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:22.367479 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c7510-cf7b-4dc4-a021-82a0ecebea5a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:22.367846 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c7510-cf7b-4dc4-a021-82a0ecebea5a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:22.368211 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c7510-cf7b-4dc4-a021-82a0ecebea5a None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:22.369044 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c7510-cf7b-4dc4-a021-82a0ecebea5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:22.369543 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c7510-cf7b-4dc4-a021-82a0ecebea5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:22.371264 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c7510-cf7b-4dc4-a021-82a0ecebea5a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:22.385687 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 565/1098] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:19:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:19:22.393838 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c7510-cf7b-4dc4-a021-82a0ecebea5a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:22.413752 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:22.476348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754f76e-2b3a-4bf4-9750-caba0cc61a24 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:22.526922 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac6a780-0e0d-4140-9f82-f155156868e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:22.527202 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c7510-cf7b-4dc4-a021-82a0ecebea5a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:22.527871 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac6a780-0e0d-4140-9f82-f155156868e0 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:22.528314 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac6a780-0e0d-4140-9f82-f155156868e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:22.528766 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac6a780-0e0d-4140-9f82-f155156868e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:22.530023 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac6a780-0e0d-4140-9f82-f155156868e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:22.530557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac6a780-0e0d-4140-9f82-f155156868e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:22.532439 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac6a780-0e0d-4140-9f82-f155156868e0 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:22.547020 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 534/1099] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:19:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:19:22.554174 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac6a780-0e0d-4140-9f82-f155156868e0 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:22.574985 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c535f-8b59-468c-8957-a3862febc46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:22.575932 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c535f-8b59-468c-8957-a3862febc46f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:22.576294 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c535f-8b59-468c-8957-a3862febc46f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:22.576747 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c535f-8b59-468c-8957-a3862febc46f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:22.577004 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:22.585601 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8c535f-8b59-468c-8957-a3862febc46f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:22.587269 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8c535f-8b59-468c-8957-a3862febc46f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:22.602925 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-7d8c535f-8b59-468c-8957-a3862febc46f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:19:22.602925 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:19:22.602925 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:19:22.602925 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:19:22.602925 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:19:22.602925 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:19:22.602925 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:19:22.602925 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:19:22.602925 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:19:22.602925 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:19:22.602925 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:19:22.602925 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:19:22.602925 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:19:22.602925 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:19:22.602925 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:19:22.604632 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:19:22.604632 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:19:22.604632 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:19:22.604632 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:19:22.604632 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:19:22.604632 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:19:22.604632 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:19:22.604632 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:19:22.604632 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:19:22.604632 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:19:22.604632 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:19:22.604632 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:19:22.604632 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:19:22.604632 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:19:22.604632 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:19:22.604632 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:19:22.604632 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:19:22.606998 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:19:22.606998 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:19:22.606998 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:19:22.606998 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:19:22.606998 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:19:22.606998 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:19:22.606998 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:19:22.606998 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:19:22.606998 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 566/1100] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:19:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:19:22.610359 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c535f-8b59-468c-8957-a3862febc46f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:22.689681 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:22.691960 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac6a780-0e0d-4140-9f82-f155156868e0 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:22.740600 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c535f-8b59-468c-8957-a3862febc46f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:22.742714 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae446e8-8a7e-4b44-98d2-d32ec5f48a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:22.743631 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae446e8-8a7e-4b44-98d2-d32ec5f48a56 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:22.743918 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae446e8-8a7e-4b44-98d2-d32ec5f48a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:22.744185 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae446e8-8a7e-4b44-98d2-d32ec5f48a56 None admin] PATH_INFO: `/v3/users/dfebd80c5e974f8b849fa63e8ef18f42` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:22.745480 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae446e8-8a7e-4b44-98d2-d32ec5f48a56 None admin] RBAC: Authorizing `identity:get_user(user_id=dfebd80c5e974f8b849fa63e8ef18f42)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:22.746974 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae446e8-8a7e-4b44-98d2-d32ec5f48a56 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:22.748669 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 535/1101] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:19:22 2022] GET /identity/v3/users/dfebd80c5e974f8b849fa63e8ef18f42 => generated 285 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:19:22.755042 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae446e8-8a7e-4b44-98d2-d32ec5f48a56 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:22.783683 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860589ca-8300-4363-ad56-c59588b460c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:22.784365 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860589ca-8300-4363-ad56-c59588b460c5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:22.784795 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860589ca-8300-4363-ad56-c59588b460c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:22.785160 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860589ca-8300-4363-ad56-c59588b460c5 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:22.786135 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860589ca-8300-4363-ad56-c59588b460c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:22.786604 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860589ca-8300-4363-ad56-c59588b460c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:22.788194 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860589ca-8300-4363-ad56-c59588b460c5 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:22.801959 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 567/1102] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 21:19:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:22.808988 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860589ca-8300-4363-ad56-c59588b460c5 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:22.859261 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:22.888109 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae446e8-8a7e-4b44-98d2-d32ec5f48a56 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:22.928373 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860589ca-8300-4363-ad56-c59588b460c5 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:22.943354 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11336edd-1bbf-4bdc-aaa5-1e7bc7267d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:22.944175 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11336edd-1bbf-4bdc-aaa5-1e7bc7267d48 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:22.944547 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11336edd-1bbf-4bdc-aaa5-1e7bc7267d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:22.944908 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11336edd-1bbf-4bdc-aaa5-1e7bc7267d48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:22.962277 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:22.969833 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eca8f5-4403-4836-b7d9-91db904ba577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:22.970652 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eca8f5-4403-4836-b7d9-91db904ba577 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:22.970997 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eca8f5-4403-4836-b7d9-91db904ba577 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:22.971329 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eca8f5-4403-4836-b7d9-91db904ba577 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:22.972452 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eca8f5-4403-4836-b7d9-91db904ba577 None admin] RBAC: Adding query filter params (role.id=bdd2ca99a0d74237b4642eae68a9a1c5, scope.project.id=c4940befa9c4405699c72deedc8a2127, user.id=e254671b21fe4661a381b9521810cf80) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:22.973926 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eca8f5-4403-4836-b7d9-91db904ba577 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:22.976405 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eca8f5-4403-4836-b7d9-91db904ba577 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:22.987529 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 568/1103] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:19:22 2022] GET /identity/v3/role_assignments?user.id=e254671b21fe4661a381b9521810cf80&scope.project.id=c4940befa9c4405699c72deedc8a2127&role.id=bdd2ca99a0d74237b4642eae68a9a1c5 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:19:23.035545 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11336edd-1bbf-4bdc-aaa5-1e7bc7267d48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:23.037192 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11336edd-1bbf-4bdc-aaa5-1e7bc7267d48 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:23.140145 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:23.160568 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 536/1104] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:19:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 21:19:23.167114 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eca8f5-4403-4836-b7d9-91db904ba577 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:23.287582 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eca8f5-4403-4836-b7d9-91db904ba577 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:23.328293 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73c313-c173-400f-ad2c-b2f2c4f03cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:23.329186 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73c313-c173-400f-ad2c-b2f2c4f03cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:23.329469 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73c313-c173-400f-ad2c-b2f2c4f03cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:23.329742 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73c313-c173-400f-ad2c-b2f2c4f03cd9 None admin] PATH_INFO: `/v3/projects/1e139d99242f43eaa802c7e7be02bdb8` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:23.331045 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73c313-c173-400f-ad2c-b2f2c4f03cd9 None admin] RBAC: Authorizing `identity:get_project(project_id=1e139d99242f43eaa802c7e7be02bdb8)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:23.333355 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73c313-c173-400f-ad2c-b2f2c4f03cd9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:23.335299 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 569/1105] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:19:23 2022] GET /identity/v3/projects/1e139d99242f43eaa802c7e7be02bdb8 => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:19:23.341676 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11336edd-1bbf-4bdc-aaa5-1e7bc7267d48 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:23.481384 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11336edd-1bbf-4bdc-aaa5-1e7bc7267d48 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:23.532736 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d0e5a-a84a-4503-bd73-a0976c8fa4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:23.533704 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d0e5a-a84a-4503-bd73-a0976c8fa4cb None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:23.534154 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d0e5a-a84a-4503-bd73-a0976c8fa4cb None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:23.535417 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d0e5a-a84a-4503-bd73-a0976c8fa4cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:23.536707 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9d0e5a-a84a-4503-bd73-a0976c8fa4cb None admin] RBAC: Adding query filter params (role.id=2b714f2e5c6c469f8fa037bda1ed16f6, scope.project.id=1e139d99242f43eaa802c7e7be02bdb8, user.id=dfebd80c5e974f8b849fa63e8ef18f42) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:23.537221 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9d0e5a-a84a-4503-bd73-a0976c8fa4cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:23.539873 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9d0e5a-a84a-4503-bd73-a0976c8fa4cb None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:23.547974 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 537/1106] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:19:23 2022] GET /identity/v3/role_assignments?user.id=dfebd80c5e974f8b849fa63e8ef18f42&scope.project.id=1e139d99242f43eaa802c7e7be02bdb8&role.id=2b714f2e5c6c469f8fa037bda1ed16f6 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:19:25.996040 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d1be3-d31b-43e0-b77b-a44fe34a6550 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:25.996669 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d1be3-d31b-43e0-b77b-a44fe34a6550 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:25.996669 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d1be3-d31b-43e0-b77b-a44fe34a6550 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:25.998014 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 570/1107] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:19:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:19:26.004181 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b5e44-58d4-4185-8fdf-decf95a5e868 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:26.004800 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b5e44-58d4-4185-8fdf-decf95a5e868 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:26.004800 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b5e44-58d4-4185-8fdf-decf95a5e868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:26.023459 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-556b5e44-58d4-4185-8fdf-decf95a5e868 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:19:26.024694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-556b5e44-58d4-4185-8fdf-decf95a5e868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:19:26.076520 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:26.104347 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 538/1108] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:19:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:19:26.130172 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32d1be3-d31b-43e0-b77b-a44fe34a6550 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:26.199903 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:26.244192 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32d1be3-d31b-43e0-b77b-a44fe34a6550 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:26.290816 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59070912-8b28-44f4-9310-4f65b8cfa660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:26.291679 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59070912-8b28-44f4-9310-4f65b8cfa660 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:26.292007 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59070912-8b28-44f4-9310-4f65b8cfa660 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:26.292324 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59070912-8b28-44f4-9310-4f65b8cfa660 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:26.293623 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59070912-8b28-44f4-9310-4f65b8cfa660 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:26.295171 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59070912-8b28-44f4-9310-4f65b8cfa660 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:26.296893 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 571/1109] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:19:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:26.303385 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b5e44-58d4-4185-8fdf-decf95a5e868 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:26.411442 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b5e44-58d4-4185-8fdf-decf95a5e868 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:26.448758 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b60a2e-7737-4ee8-a556-f985bf21c352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:26.449442 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b60a2e-7737-4ee8-a556-f985bf21c352 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:26.449716 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b60a2e-7737-4ee8-a556-f985bf21c352 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:26.449990 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b60a2e-7737-4ee8-a556-f985bf21c352 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:26.450745 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b60a2e-7737-4ee8-a556-f985bf21c352 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:26.452210 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b60a2e-7737-4ee8-a556-f985bf21c352 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:26.480041 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 539/1110] 10.222.0.65 () {60 vars in 1132 bytes} [Thu May 19 21:19:26 2022] POST /identity/v3/users => generated 286 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:19:28.841452 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f665e0d-0f48-4de7-a71b-4eddfc8d559c None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:28.842048 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f665e0d-0f48-4de7-a71b-4eddfc8d559c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:28.842048 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f665e0d-0f48-4de7-a71b-4eddfc8d559c None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:28.843922 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 572/1111] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:19:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:19:28.851712 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf1a78-e1d8-467d-961e-5ca491c29add None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:28.852283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf1a78-e1d8-467d-961e-5ca491c29add None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:28.852283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf1a78-e1d8-467d-961e-5ca491c29add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:28.873868 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-93bf1a78-e1d8-467d-961e-5ca491c29add None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:19:28.875105 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-93bf1a78-e1d8-467d-961e-5ca491c29add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:19:28.905533 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:28.951143 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 540/1112] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:19:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:19:28.975375 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f665e0d-0f48-4de7-a71b-4eddfc8d559c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:29.097243 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f665e0d-0f48-4de7-a71b-4eddfc8d559c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:29.130100 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:29.137081 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f22be7-b367-4fff-a030-84e2a1c52aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:29.137961 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f22be7-b367-4fff-a030-84e2a1c52aef None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:29.138272 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f22be7-b367-4fff-a030-84e2a1c52aef None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:29.138561 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f22be7-b367-4fff-a030-84e2a1c52aef None admin] PATH_INFO: `/v3/roles/566c421659204df185f8b77bf3bb3afc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:29.140102 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f22be7-b367-4fff-a030-84e2a1c52aef None admin] RBAC: Authorizing `identity:get_role(role_id=566c421659204df185f8b77bf3bb3afc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:29.141481 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f22be7-b367-4fff-a030-84e2a1c52aef None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:29.142906 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 573/1113] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:19:28 2022] GET /identity/v3/roles/566c421659204df185f8b77bf3bb3afc => generated 223 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:29.149508 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf1a78-e1d8-467d-961e-5ca491c29add None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:29.305183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf1a78-e1d8-467d-961e-5ca491c29add None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:29.367921 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987b937-9e64-42ae-ab9e-a9868dd88e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:29.369363 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987b937-9e64-42ae-ab9e-a9868dd88e9b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:29.369854 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987b937-9e64-42ae-ab9e-a9868dd88e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:29.370375 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987b937-9e64-42ae-ab9e-a9868dd88e9b None admin] PATH_INFO: `/v3/users/f160695e2f30470389555033978c994f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:29.395569 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d987b937-9e64-42ae-ab9e-a9868dd88e9b None admin] RBAC: Authorizing `identity:get_user(user_id=f160695e2f30470389555033978c994f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:29.398289 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d987b937-9e64-42ae-ab9e-a9868dd88e9b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:29.401546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 541/1114] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:19:29 2022] GET /identity/v3/users/f160695e2f30470389555033978c994f => generated 286 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:29.407989 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f22be7-b367-4fff-a030-84e2a1c52aef None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:29.526297 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f22be7-b367-4fff-a030-84e2a1c52aef None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:29.579365 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8aece-43ff-4cb4-bc88-50ada81a4494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:29.580043 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8aece-43ff-4cb4-bc88-50ada81a4494 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:29.580330 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8aece-43ff-4cb4-bc88-50ada81a4494 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:29.580605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8aece-43ff-4cb4-bc88-50ada81a4494 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:29.654800 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:29.659087 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b8aece-43ff-4cb4-bc88-50ada81a4494 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:29.661111 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b8aece-43ff-4cb4-bc88-50ada81a4494 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:29.772667 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 574/1115] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:19:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 21:19:29.781831 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987b937-9e64-42ae-ab9e-a9868dd88e9b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:29.808493 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:29.903059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987b937-9e64-42ae-ab9e-a9868dd88e9b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:29.944987 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b8baad-3f44-4d76-9015-9183a150607d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:29.945771 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8baad-3f44-4d76-9015-9183a150607d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:29.946055 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8baad-3f44-4d76-9015-9183a150607d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:29.946343 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8baad-3f44-4d76-9015-9183a150607d None admin] PATH_INFO: `/v3/projects/1e139d99242f43eaa802c7e7be02bdb8` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:29.947599 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b8baad-3f44-4d76-9015-9183a150607d None admin] RBAC: Authorizing `identity:get_project(project_id=1e139d99242f43eaa802c7e7be02bdb8)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:29.949069 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b8baad-3f44-4d76-9015-9183a150607d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:29.950839 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 542/1116] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:19:29 2022] GET /identity/v3/projects/1e139d99242f43eaa802c7e7be02bdb8 => generated 311 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:29.961222 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8aece-43ff-4cb4-bc88-50ada81a4494 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:30.153416 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:30.160872 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8aece-43ff-4cb4-bc88-50ada81a4494 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:30.221544 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f8491-e025-4648-a393-c920520b95f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:30.222728 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f8491-e025-4648-a393-c920520b95f1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:30.223228 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f8491-e025-4648-a393-c920520b95f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:30.223733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f8491-e025-4648-a393-c920520b95f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:30.225472 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f8491-e025-4648-a393-c920520b95f1 None admin] RBAC: Adding query filter params (role.id=566c421659204df185f8b77bf3bb3afc, scope.project.id=1e139d99242f43eaa802c7e7be02bdb8, user.id=f160695e2f30470389555033978c994f) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:30.226036 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f8491-e025-4648-a393-c920520b95f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:30.228938 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f8491-e025-4648-a393-c920520b95f1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:30.241272 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 575/1117] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:19:29 2022] GET /identity/v3/role_assignments?user.id=f160695e2f30470389555033978c994f&scope.project.id=1e139d99242f43eaa802c7e7be02bdb8&role.id=566c421659204df185f8b77bf3bb3afc => generated 259 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:32.528409 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623161bc-6c08-4be6-8c49-fa9ac34ae001 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:32.529011 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623161bc-6c08-4be6-8c49-fa9ac34ae001 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:32.529365 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623161bc-6c08-4be6-8c49-fa9ac34ae001 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:32.531982 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 543/1118] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:19:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:19:32.542114 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc18af-6830-433e-ae7a-a65080610e1e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:32.542755 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc18af-6830-433e-ae7a-a65080610e1e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:32.543248 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc18af-6830-433e-ae7a-a65080610e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:32.571825 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-7bcc18af-6830-433e-ae7a-a65080610e1e None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:19:32.573870 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7bcc18af-6830-433e-ae7a-a65080610e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:19:32.700405 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 576/1119] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:19:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:19:32.740244 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623161bc-6c08-4be6-8c49-fa9ac34ae001 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:32.848483 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:32.867192 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623161bc-6c08-4be6-8c49-fa9ac34ae001 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:32.929562 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce74dc-db85-4d78-84f1-e4f9eeda9957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:32.931092 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce74dc-db85-4d78-84f1-e4f9eeda9957 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:32.931522 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce74dc-db85-4d78-84f1-e4f9eeda9957 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:32.931994 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce74dc-db85-4d78-84f1-e4f9eeda9957 None admin] PATH_INFO: `/v3/roles/566c421659204df185f8b77bf3bb3afc` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:32.934741 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce74dc-db85-4d78-84f1-e4f9eeda9957 None admin] RBAC: Authorizing `identity:get_role(role_id=566c421659204df185f8b77bf3bb3afc)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:32.937236 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce74dc-db85-4d78-84f1-e4f9eeda9957 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:32.939868 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 544/1120] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:19:32 2022] GET /identity/v3/roles/566c421659204df185f8b77bf3bb3afc => generated 223 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:32.946204 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc18af-6830-433e-ae7a-a65080610e1e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:33.045627 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:33.056278 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc18af-6830-433e-ae7a-a65080610e1e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:33.095744 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e415eee-12c4-496e-be07-94fdea45200c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:33.096632 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e415eee-12c4-496e-be07-94fdea45200c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:33.097144 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e415eee-12c4-496e-be07-94fdea45200c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:33.097498 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e415eee-12c4-496e-be07-94fdea45200c None admin] PATH_INFO: `/v3/users/f160695e2f30470389555033978c994f` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:33.098766 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e415eee-12c4-496e-be07-94fdea45200c None admin] RBAC: Authorizing `identity:get_user(user_id=f160695e2f30470389555033978c994f)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:33.100255 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e415eee-12c4-496e-be07-94fdea45200c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:33.101988 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 577/1121] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:19:32 2022] GET /identity/v3/users/f160695e2f30470389555033978c994f => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:33.108372 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce74dc-db85-4d78-84f1-e4f9eeda9957 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:33.219745 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce74dc-db85-4d78-84f1-e4f9eeda9957 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:33.258180 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b91e472-d885-4b47-af57-3ad7cac9846a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:33.259106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b91e472-d885-4b47-af57-3ad7cac9846a None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:33.259390 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b91e472-d885-4b47-af57-3ad7cac9846a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:33.259665 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b91e472-d885-4b47-af57-3ad7cac9846a None admin] PATH_INFO: `/v3/projects/1e139d99242f43eaa802c7e7be02bdb8` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:33.260928 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b91e472-d885-4b47-af57-3ad7cac9846a None admin] RBAC: Authorizing `identity:get_project(project_id=1e139d99242f43eaa802c7e7be02bdb8)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:33.262447 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b91e472-d885-4b47-af57-3ad7cac9846a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:33.264182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 545/1122] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:19:33 2022] GET /identity/v3/projects/1e139d99242f43eaa802c7e7be02bdb8 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:33.270608 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e415eee-12c4-496e-be07-94fdea45200c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:33.378284 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e415eee-12c4-496e-be07-94fdea45200c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:33.415335 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e3e26-d7c2-497f-b0fa-f0814bfea037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:33.416236 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e3e26-d7c2-497f-b0fa-f0814bfea037 None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:33.416518 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e3e26-d7c2-497f-b0fa-f0814bfea037 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:33.416800 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e3e26-d7c2-497f-b0fa-f0814bfea037 None admin] PATH_INFO: `/v3/projects/1e139d99242f43eaa802c7e7be02bdb8/users/f160695e2f30470389555033978c994f/roles/566c421659204df185f8b77bf3bb3afc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:33.419107 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e3e26-d7c2-497f-b0fa-f0814bfea037 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e139d99242f43eaa802c7e7be02bdb8, user_id=f160695e2f30470389555033978c994f, role_id=566c421659204df185f8b77bf3bb3afc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:33.420570 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e3e26-d7c2-497f-b0fa-f0814bfea037 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:33.434839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 578/1123] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 21:19:33 2022] PUT /identity/v3/projects/1e139d99242f43eaa802c7e7be02bdb8/users/f160695e2f30470389555033978c994f/roles/566c421659204df185f8b77bf3bb3afc => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:19:35.740924 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64be11-56e9-411c-8958-21bf9293a5f0 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:35.741539 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64be11-56e9-411c-8958-21bf9293a5f0 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:35.741539 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64be11-56e9-411c-8958-21bf9293a5f0 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:35.743289 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 546/1124] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:19:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:19:35.749842 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839125b-0364-4610-8f7e-4eee7cee37d5 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:35.750514 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839125b-0364-4610-8f7e-4eee7cee37d5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:35.750514 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839125b-0364-4610-8f7e-4eee7cee37d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:35.770462 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-3839125b-0364-4610-8f7e-4eee7cee37d5 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:19:35.771800 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3839125b-0364-4610-8f7e-4eee7cee37d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:19:35.789724 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:35.856051 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 579/1125] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:19:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:19:35.881502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64be11-56e9-411c-8958-21bf9293a5f0 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:35.914897 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:36.006853 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64be11-56e9-411c-8958-21bf9293a5f0 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:36.048374 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a035c7-dd6d-4159-a364-07042893d394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:36.049254 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a035c7-dd6d-4159-a364-07042893d394 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:36.049534 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a035c7-dd6d-4159-a364-07042893d394 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:36.049798 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a035c7-dd6d-4159-a364-07042893d394 None admin] PATH_INFO: `/v3/roles/566c421659204df185f8b77bf3bb3afc` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:36.051757 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a035c7-dd6d-4159-a364-07042893d394 None admin] RBAC: Authorizing `identity:get_role(role_id=566c421659204df185f8b77bf3bb3afc)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:36.053174 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a035c7-dd6d-4159-a364-07042893d394 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:36.054665 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 547/1126] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:19:35 2022] GET /identity/v3/roles/566c421659204df185f8b77bf3bb3afc => generated 223 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:36.061228 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3839125b-0364-4610-8f7e-4eee7cee37d5 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:36.176411 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3839125b-0364-4610-8f7e-4eee7cee37d5 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:36.214483 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed0db1c-7bb1-43db-9265-6e2157548db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:36.215372 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed0db1c-7bb1-43db-9265-6e2157548db3 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:36.215652 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed0db1c-7bb1-43db-9265-6e2157548db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:36.215925 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed0db1c-7bb1-43db-9265-6e2157548db3 None admin] PATH_INFO: `/v3/users/f160695e2f30470389555033978c994f` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:36.217224 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed0db1c-7bb1-43db-9265-6e2157548db3 None admin] RBAC: Authorizing `identity:get_user(user_id=f160695e2f30470389555033978c994f)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:36.218707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed0db1c-7bb1-43db-9265-6e2157548db3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:36.220380 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 580/1127] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:19:36 2022] GET /identity/v3/users/f160695e2f30470389555033978c994f => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:36.226968 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a035c7-dd6d-4159-a364-07042893d394 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:36.330679 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:36.343830 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a035c7-dd6d-4159-a364-07042893d394 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:36.407012 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e000cc-7f96-469c-ac03-f893ee61dd7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:36.408159 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e000cc-7f96-469c-ac03-f893ee61dd7b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:36.408660 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e000cc-7f96-469c-ac03-f893ee61dd7b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:36.409146 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e000cc-7f96-469c-ac03-f893ee61dd7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:36.535380 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e000cc-7f96-469c-ac03-f893ee61dd7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:36.538112 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e000cc-7f96-469c-ac03-f893ee61dd7b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:36.717939 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:36.727931 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 548/1128] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:19:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 21:19:36.734519 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed0db1c-7bb1-43db-9265-6e2157548db3 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:36.757273 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:36.847533 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed0db1c-7bb1-43db-9265-6e2157548db3 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:36.903986 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bf80e-4dd2-4bcb-bb31-51227413c132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:36.904865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bf80e-4dd2-4bcb-bb31-51227413c132 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:36.905149 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bf80e-4dd2-4bcb-bb31-51227413c132 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:36.905427 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bf80e-4dd2-4bcb-bb31-51227413c132 None admin] PATH_INFO: `/v3/projects/1e139d99242f43eaa802c7e7be02bdb8` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:36.906713 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58bf80e-4dd2-4bcb-bb31-51227413c132 None admin] RBAC: Authorizing `identity:get_project(project_id=1e139d99242f43eaa802c7e7be02bdb8)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:36.908204 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58bf80e-4dd2-4bcb-bb31-51227413c132 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:36.910107 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 581/1129] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:19:36 2022] GET /identity/v3/projects/1e139d99242f43eaa802c7e7be02bdb8 => generated 311 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:36.920497 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e000cc-7f96-469c-ac03-f893ee61dd7b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:37.100871 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e000cc-7f96-469c-ac03-f893ee61dd7b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:37.165852 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4ecc9-1932-41b7-b6da-701f0bd05303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:37.167010 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4ecc9-1932-41b7-b6da-701f0bd05303 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:37.167503 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4ecc9-1932-41b7-b6da-701f0bd05303 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:37.167982 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4ecc9-1932-41b7-b6da-701f0bd05303 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:37.169556 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4ecc9-1932-41b7-b6da-701f0bd05303 None admin] RBAC: Adding query filter params (role.id=566c421659204df185f8b77bf3bb3afc, scope.project.id=1e139d99242f43eaa802c7e7be02bdb8, user.id=f160695e2f30470389555033978c994f) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:37.170102 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4ecc9-1932-41b7-b6da-701f0bd05303 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:37.172787 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4ecc9-1932-41b7-b6da-701f0bd05303 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:37.185189 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 549/1130] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:19:36 2022] GET /identity/v3/role_assignments?user.id=f160695e2f30470389555033978c994f&scope.project.id=1e139d99242f43eaa802c7e7be02bdb8&role.id=566c421659204df185f8b77bf3bb3afc => generated 608 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:39.497004 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e84f9-e7e8-4e4b-827e-57081fc44929 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:39.497660 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e84f9-e7e8-4e4b-827e-57081fc44929 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:39.497660 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e84f9-e7e8-4e4b-827e-57081fc44929 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:39.499005 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 582/1131] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:19:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:19:39.505498 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069224c-e55e-4ba3-87e9-5a74ca0a5425 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:39.506076 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069224c-e55e-4ba3-87e9-5a74ca0a5425 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:39.506076 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069224c-e55e-4ba3-87e9-5a74ca0a5425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:39.526224 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-c069224c-e55e-4ba3-87e9-5a74ca0a5425 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:19:39.527524 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c069224c-e55e-4ba3-87e9-5a74ca0a5425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:19:39.591959 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:39.608476 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 550/1132] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:19:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:19:39.633547 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e84f9-e7e8-4e4b-827e-57081fc44929 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:39.737310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:39.758625 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e84f9-e7e8-4e4b-827e-57081fc44929 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:39.800034 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95117f8-fdba-4a3f-a754-5b8c0ac412a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:39.801064 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95117f8-fdba-4a3f-a754-5b8c0ac412a7 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:39.801350 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95117f8-fdba-4a3f-a754-5b8c0ac412a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:39.801623 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95117f8-fdba-4a3f-a754-5b8c0ac412a7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:39.802975 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95117f8-fdba-4a3f-a754-5b8c0ac412a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:39.804453 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95117f8-fdba-4a3f-a754-5b8c0ac412a7 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:39.806188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 583/1133] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:19:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:39.812639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069224c-e55e-4ba3-87e9-5a74ca0a5425 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:39.924189 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069224c-e55e-4ba3-87e9-5a74ca0a5425 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:39.964759 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5990d6db-a2fc-42f6-a37e-e75c5fe12f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:39.965487 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990d6db-a2fc-42f6-a37e-e75c5fe12f46 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:39.965823 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990d6db-a2fc-42f6-a37e-e75c5fe12f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:39.966146 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990d6db-a2fc-42f6-a37e-e75c5fe12f46 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:39.966965 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5990d6db-a2fc-42f6-a37e-e75c5fe12f46 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:39.968476 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5990d6db-a2fc-42f6-a37e-e75c5fe12f46 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:39.997831 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 551/1134] 10.222.0.65 () {60 vars in 1141 bytes} [Thu May 19 21:19:39 2022] POST /identity/v3/projects => generated 311 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:19:42.365994 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f04693-6b61-497f-a39f-a2db010028a9 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:42.366648 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f04693-6b61-497f-a39f-a2db010028a9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:42.366648 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f04693-6b61-497f-a39f-a2db010028a9 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:42.367984 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 584/1135] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:19:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:19:42.376011 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b2efc-8c37-4638-b297-6b51ada78ab1 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:42.376586 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b2efc-8c37-4638-b297-6b51ada78ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:42.376586 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b2efc-8c37-4638-b297-6b51ada78ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:42.395573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-271b2efc-8c37-4638-b297-6b51ada78ab1 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:19:42.396812 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-271b2efc-8c37-4638-b297-6b51ada78ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:19:42.436306 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:42.475582 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 552/1136] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:19:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:19:42.501997 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f04693-6b61-497f-a39f-a2db010028a9 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:42.621938 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f04693-6b61-497f-a39f-a2db010028a9 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:42.659423 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7269736-e33b-4310-81ff-46400298c2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:42.660239 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7269736-e33b-4310-81ff-46400298c2de None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:42.660520 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7269736-e33b-4310-81ff-46400298c2de None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:42.660787 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7269736-e33b-4310-81ff-46400298c2de None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:42.662007 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7269736-e33b-4310-81ff-46400298c2de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:42.663502 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7269736-e33b-4310-81ff-46400298c2de None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:42.665180 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 585/1137] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:19:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:42.671262 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271b2efc-8c37-4638-b297-6b51ada78ab1 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:42.777422 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271b2efc-8c37-4638-b297-6b51ada78ab1 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:42.813798 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ada53-0b20-4258-b86e-4f39f9759411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:42.814510 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ada53-0b20-4258-b86e-4f39f9759411 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:42.814790 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ada53-0b20-4258-b86e-4f39f9759411 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:42.815119 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ada53-0b20-4258-b86e-4f39f9759411 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:42.815854 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7ada53-0b20-4258-b86e-4f39f9759411 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:42.817330 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7ada53-0b20-4258-b86e-4f39f9759411 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:42.840120 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 553/1138] 10.222.0.65 () {60 vars in 1132 bytes} [Thu May 19 21:19:42 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:19:45.197262 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62efb3-ec8b-46d1-b380-3734118c7574 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:45.197834 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62efb3-ec8b-46d1-b380-3734118c7574 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:45.197834 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62efb3-ec8b-46d1-b380-3734118c7574 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:45.199110 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 586/1139] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:19:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:19:45.205262 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5e04d-5bc5-4a30-a934-a0bbfed9e044 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:45.205873 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5e04d-5bc5-4a30-a934-a0bbfed9e044 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:45.206813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5e04d-5bc5-4a30-a934-a0bbfed9e044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:45.225467 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-43b5e04d-5bc5-4a30-a934-a0bbfed9e044 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:19:45.226736 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-43b5e04d-5bc5-4a30-a934-a0bbfed9e044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:19:45.252210 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:45.313111 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 554/1140] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:19:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:19:45.351824 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a62efb3-ec8b-46d1-b380-3734118c7574 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:45.385660 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:45.491781 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a62efb3-ec8b-46d1-b380-3734118c7574 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:45.531929 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171fecb-bb52-44cc-be5a-56d677541612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:45.532750 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171fecb-bb52-44cc-be5a-56d677541612 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:45.533032 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171fecb-bb52-44cc-be5a-56d677541612 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:45.533302 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171fecb-bb52-44cc-be5a-56d677541612 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:45.546133 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9171fecb-bb52-44cc-be5a-56d677541612 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:19:45.554131 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-9171fecb-bb52-44cc-be5a-56d677541612 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:19:45.554131 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:19:45.554131 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:19:45.554131 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:19:45.554131 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:19:45.554131 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:19:45.554131 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:19:45.554131 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:19:45.554131 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:19:45.554131 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:19:45.554131 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:19:45.554131 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:19:45.554131 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:19:45.554131 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:19:45.554131 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:19:45.554131 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:19:45.556063 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:19:45.556063 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:19:45.556063 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:19:45.556063 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:19:45.556063 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:19:45.556063 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:19:45.556063 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:19:45.556063 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:19:45.556063 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:19:45.556063 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:19:45.556063 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:19:45.556063 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:19:45.556063 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:19:45.556063 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:19:45.556063 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:19:45.556063 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:19:45.558067 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:19:45.558067 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:19:45.558067 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:19:45.558067 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:19:45.558067 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:19:45.558067 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:19:45.558067 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:19:45.558067 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:19:45.558067 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:19:45.558067 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:19:45.558067 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:19:45.558067 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:19:45.558067 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 587/1141] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 21:19:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:19:45.563529 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5e04d-5bc5-4a30-a934-a0bbfed9e044 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:45.678704 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5e04d-5bc5-4a30-a934-a0bbfed9e044 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:45.723164 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1079eb29-e5ea-4358-9a14-ea6bcf232322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:45.723864 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1079eb29-e5ea-4358-9a14-ea6bcf232322 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:45.724144 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1079eb29-e5ea-4358-9a14-ea6bcf232322 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:45.724421 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1079eb29-e5ea-4358-9a14-ea6bcf232322 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:45.725196 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1079eb29-e5ea-4358-9a14-ea6bcf232322 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:45.725505 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1079eb29-e5ea-4358-9a14-ea6bcf232322 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:45.727048 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1079eb29-e5ea-4358-9a14-ea6bcf232322 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:45.736882 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 555/1142] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:19:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:45.743636 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171fecb-bb52-44cc-be5a-56d677541612 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:45.860310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:45.883192 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171fecb-bb52-44cc-be5a-56d677541612 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:45.922625 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80694bc-5f93-424b-b2ee-c00207ca902b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:45.923533 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80694bc-5f93-424b-b2ee-c00207ca902b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:45.923818 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80694bc-5f93-424b-b2ee-c00207ca902b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:45.924093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80694bc-5f93-424b-b2ee-c00207ca902b None admin] PATH_INFO: `/v3/users/00ebba1e469c4e7bb6302d24c7b89634` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:45.941301 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80694bc-5f93-424b-b2ee-c00207ca902b None admin] RBAC: Authorizing `identity:get_user(user_id=00ebba1e469c4e7bb6302d24c7b89634)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:45.942882 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80694bc-5f93-424b-b2ee-c00207ca902b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:45.944781 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 588/1143] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:19:45 2022] GET /identity/v3/users/00ebba1e469c4e7bb6302d24c7b89634 => generated 286 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:45.954452 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1079eb29-e5ea-4358-9a14-ea6bcf232322 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:45.979849 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:46.088232 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1079eb29-e5ea-4358-9a14-ea6bcf232322 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:46.132845 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d0ff37-e652-42ee-9418-f3a49444aef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:46.133532 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0ff37-e652-42ee-9418-f3a49444aef2 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:46.133810 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0ff37-e652-42ee-9418-f3a49444aef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:46.134083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0ff37-e652-42ee-9418-f3a49444aef2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:46.222575 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d0ff37-e652-42ee-9418-f3a49444aef2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:46.224077 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d0ff37-e652-42ee-9418-f3a49444aef2 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:46.245602 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:46.372238 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 556/1144] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:19:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 21:19:46.379583 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80694bc-5f93-424b-b2ee-c00207ca902b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:46.497387 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80694bc-5f93-424b-b2ee-c00207ca902b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:46.538192 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551acb89-eb68-4efd-a416-14221356ea06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:46.539262 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551acb89-eb68-4efd-a416-14221356ea06 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:46.539539 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551acb89-eb68-4efd-a416-14221356ea06 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:46.539818 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551acb89-eb68-4efd-a416-14221356ea06 None admin] PATH_INFO: `/v3/projects/db723326dd804ca6a85aeb274243d246` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:46.541194 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551acb89-eb68-4efd-a416-14221356ea06 None admin] RBAC: Authorizing `identity:get_project(project_id=db723326dd804ca6a85aeb274243d246)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:46.542707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551acb89-eb68-4efd-a416-14221356ea06 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:46.544487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 589/1145] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:19:46 2022] GET /identity/v3/projects/db723326dd804ca6a85aeb274243d246 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:46.551168 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d0ff37-e652-42ee-9418-f3a49444aef2 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:46.663949 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d0ff37-e652-42ee-9418-f3a49444aef2 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:46.682650 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:46.702000 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c954a521-754c-49e7-a591-977189a98fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:46.702864 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954a521-754c-49e7-a591-977189a98fcf None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:46.703173 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954a521-754c-49e7-a591-977189a98fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:46.703489 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954a521-754c-49e7-a591-977189a98fcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:46.704387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c954a521-754c-49e7-a591-977189a98fcf None admin] RBAC: Adding query filter params (role.id=2b714f2e5c6c469f8fa037bda1ed16f6, scope.project.id=db723326dd804ca6a85aeb274243d246, user.id=00ebba1e469c4e7bb6302d24c7b89634) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:46.704740 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c954a521-754c-49e7-a591-977189a98fcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:46.706607 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c954a521-754c-49e7-a591-977189a98fcf None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:46.713475 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 557/1146] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:19:46 2022] GET /identity/v3/role_assignments?user.id=00ebba1e469c4e7bb6302d24c7b89634&scope.project.id=db723326dd804ca6a85aeb274243d246&role.id=2b714f2e5c6c469f8fa037bda1ed16f6 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:49.066006 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af627a1-9b88-4b42-9e35-0427c153d03f None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:49.066648 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af627a1-9b88-4b42-9e35-0427c153d03f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:49.066648 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af627a1-9b88-4b42-9e35-0427c153d03f None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:49.067953 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 590/1147] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:19:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:19:49.074126 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c459a49-5b5b-4c73-9e23-c2978e4d4ba1 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:49.074737 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c459a49-5b5b-4c73-9e23-c2978e4d4ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:49.074737 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c459a49-5b5b-4c73-9e23-c2978e4d4ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:49.093061 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-1c459a49-5b5b-4c73-9e23-c2978e4d4ba1 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:19:49.094310 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1c459a49-5b5b-4c73-9e23-c2978e4d4ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:19:49.170963 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 558/1148] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:19:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:19:49.194280 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af627a1-9b88-4b42-9e35-0427c153d03f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:49.207231 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:49.305949 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af627a1-9b88-4b42-9e35-0427c153d03f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:49.344284 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02998dd-2b21-4c71-ae5e-571b67f08d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:49.345165 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02998dd-2b21-4c71-ae5e-571b67f08d11 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:49.345489 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02998dd-2b21-4c71-ae5e-571b67f08d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:49.345806 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02998dd-2b21-4c71-ae5e-571b67f08d11 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:49.357576 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02998dd-2b21-4c71-ae5e-571b67f08d11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:19:49.365983 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-b02998dd-2b21-4c71-ae5e-571b67f08d11 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:19:49.365983 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:19:49.365983 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:19:49.365983 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:19:49.365983 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:19:49.365983 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:19:49.365983 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:19:49.365983 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:19:49.365983 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:19:49.365983 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:19:49.365983 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:19:49.365983 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:19:49.365983 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:19:49.365983 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:19:49.365983 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:19:49.365983 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:19:49.367707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:19:49.367707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:19:49.367707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:19:49.367707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:19:49.367707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:19:49.367707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:19:49.367707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:19:49.367707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:19:49.367707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:19:49.367707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:19:49.367707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:19:49.367707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:19:49.367707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:19:49.367707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:19:49.367707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:19:49.367707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:19:49.368669 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:19:49.368669 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:19:49.368669 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:19:49.368669 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:19:49.368669 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:19:49.368669 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:19:49.368669 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:19:49.368669 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:19:49.368669 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:19:49.368669 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:19:49.368669 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:19:49.368669 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:19:49.368669 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 591/1149] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 21:19:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:19:49.376723 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c459a49-5b5b-4c73-9e23-c2978e4d4ba1 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:49.490134 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c459a49-5b5b-4c73-9e23-c2978e4d4ba1 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:49.505086 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:49.529835 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7046e-d278-499c-8690-473f93306159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:49.530597 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7046e-d278-499c-8690-473f93306159 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:49.530921 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7046e-d278-499c-8690-473f93306159 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:49.531240 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7046e-d278-499c-8690-473f93306159 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:49.532045 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7046e-d278-499c-8690-473f93306159 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:49.532398 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7046e-d278-499c-8690-473f93306159 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:49.533922 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7046e-d278-499c-8690-473f93306159 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:49.543795 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 559/1150] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:19:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:19:49.550041 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02998dd-2b21-4c71-ae5e-571b67f08d11 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:49.623333 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:49.661384 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02998dd-2b21-4c71-ae5e-571b67f08d11 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:49.700539 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003328a-6be8-4dfa-8749-2179d72e1bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:49.701433 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003328a-6be8-4dfa-8749-2179d72e1bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:49.701713 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003328a-6be8-4dfa-8749-2179d72e1bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:49.701989 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003328a-6be8-4dfa-8749-2179d72e1bd4 None admin] PATH_INFO: `/v3/users/00ebba1e469c4e7bb6302d24c7b89634` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:49.703261 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c003328a-6be8-4dfa-8749-2179d72e1bd4 None admin] RBAC: Authorizing `identity:get_user(user_id=00ebba1e469c4e7bb6302d24c7b89634)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:49.704765 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c003328a-6be8-4dfa-8749-2179d72e1bd4 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:49.706720 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 592/1151] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:19:49 2022] GET /identity/v3/users/00ebba1e469c4e7bb6302d24c7b89634 => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:49.713204 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7046e-d278-499c-8690-473f93306159 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:49.825691 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7046e-d278-499c-8690-473f93306159 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:49.865749 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94423c6-eaf9-447c-8876-d58b4b26cca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:49.866718 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94423c6-eaf9-447c-8876-d58b4b26cca8 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:49.866997 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94423c6-eaf9-447c-8876-d58b4b26cca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:49.867272 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94423c6-eaf9-447c-8876-d58b4b26cca8 None admin] PATH_INFO: `/v3/projects/db723326dd804ca6a85aeb274243d246` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:49.868757 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94423c6-eaf9-447c-8876-d58b4b26cca8 None admin] RBAC: Authorizing `identity:get_project(project_id=db723326dd804ca6a85aeb274243d246)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:49.870344 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94423c6-eaf9-447c-8876-d58b4b26cca8 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:49.872280 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 560/1152] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:19:49 2022] GET /identity/v3/projects/db723326dd804ca6a85aeb274243d246 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:49.879544 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003328a-6be8-4dfa-8749-2179d72e1bd4 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:49.998969 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003328a-6be8-4dfa-8749-2179d72e1bd4 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:50.036672 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a5600-f87a-4b67-a505-1a90adae069e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:50.037652 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a5600-f87a-4b67-a505-1a90adae069e None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:50.037972 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a5600-f87a-4b67-a505-1a90adae069e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:50.038311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a5600-f87a-4b67-a505-1a90adae069e None admin] PATH_INFO: `/v3/projects/db723326dd804ca6a85aeb274243d246/users/00ebba1e469c4e7bb6302d24c7b89634/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:50.040498 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357a5600-f87a-4b67-a505-1a90adae069e None admin] RBAC: Authorizing `identity:create_grant(project_id=db723326dd804ca6a85aeb274243d246, user_id=00ebba1e469c4e7bb6302d24c7b89634, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:50.042002 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357a5600-f87a-4b67-a505-1a90adae069e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:50.045925 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:50.060411 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 593/1153] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 21:19:49 2022] PUT /identity/v3/projects/db723326dd804ca6a85aeb274243d246/users/00ebba1e469c4e7bb6302d24c7b89634/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:19:52.328033 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d49f8e-9366-4b0e-b838-dc49e1d6099c None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:52.328612 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d49f8e-9366-4b0e-b838-dc49e1d6099c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:52.328612 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d49f8e-9366-4b0e-b838-dc49e1d6099c None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:52.330111 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 561/1154] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:19:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:19:52.336842 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c1d00-39cc-4c38-be07-80bb71c4dc3e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:52.337460 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c1d00-39cc-4c38-be07-80bb71c4dc3e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:52.337460 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c1d00-39cc-4c38-be07-80bb71c4dc3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:52.357258 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-da4c1d00-39cc-4c38-be07-80bb71c4dc3e None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:19:52.358567 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-da4c1d00-39cc-4c38-be07-80bb71c4dc3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:19:52.447347 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 594/1155] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:19:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:19:52.472230 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d49f8e-9366-4b0e-b838-dc49e1d6099c None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:52.532998 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:52.614941 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d49f8e-9366-4b0e-b838-dc49e1d6099c None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:52.658120 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41936278-3ac8-42e4-9364-ed85f0bf9310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:52.659724 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41936278-3ac8-42e4-9364-ed85f0bf9310 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:52.660055 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41936278-3ac8-42e4-9364-ed85f0bf9310 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:52.660389 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41936278-3ac8-42e4-9364-ed85f0bf9310 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:52.673239 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41936278-3ac8-42e4-9364-ed85f0bf9310 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:19:52.681938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-41936278-3ac8-42e4-9364-ed85f0bf9310 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:19:52.681938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:19:52.681938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:19:52.681938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:19:52.681938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:19:52.681938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:19:52.681938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:19:52.681938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:19:52.681938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:19:52.681938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:19:52.681938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:19:52.681938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:19:52.681938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:19:52.681938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:19:52.681938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:19:52.681938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:19:52.683249 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:19:52.683249 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:19:52.683249 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:19:52.683249 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:19:52.683249 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:19:52.683249 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:19:52.683249 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:19:52.683249 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:19:52.683249 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:19:52.683249 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:19:52.683249 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:19:52.683249 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:19:52.683249 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:19:52.683249 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:19:52.683249 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:19:52.683249 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:19:52.684782 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:19:52.684782 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:19:52.684782 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:19:52.684782 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:19:52.684782 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:19:52.684782 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:19:52.684782 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:19:52.684782 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:19:52.684782 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:19:52.684782 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:19:52.684782 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:19:52.684782 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:19:52.684782 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 562/1156] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 21:19:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:19:52.689864 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4c1d00-39cc-4c38-be07-80bb71c4dc3e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:52.810169 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4c1d00-39cc-4c38-be07-80bb71c4dc3e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:52.841399 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:52.852188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785227c2-0e97-491d-adc4-21ca7f988e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:52.852937 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785227c2-0e97-491d-adc4-21ca7f988e9c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:52.853294 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785227c2-0e97-491d-adc4-21ca7f988e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:52.853632 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785227c2-0e97-491d-adc4-21ca7f988e9c None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:52.854519 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785227c2-0e97-491d-adc4-21ca7f988e9c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:52.854886 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785227c2-0e97-491d-adc4-21ca7f988e9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:52.856345 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785227c2-0e97-491d-adc4-21ca7f988e9c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:52.867088 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 595/1157] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:19:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:52.873292 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41936278-3ac8-42e4-9364-ed85f0bf9310 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:52.998494 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41936278-3ac8-42e4-9364-ed85f0bf9310 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:53.000365 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:53.041058 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d84cba9-0081-4cf9-874d-012eac357917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:53.041981 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84cba9-0081-4cf9-874d-012eac357917 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:53.042282 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84cba9-0081-4cf9-874d-012eac357917 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:53.042578 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84cba9-0081-4cf9-874d-012eac357917 None admin] PATH_INFO: `/v3/users/00ebba1e469c4e7bb6302d24c7b89634` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:53.043963 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d84cba9-0081-4cf9-874d-012eac357917 None admin] RBAC: Authorizing `identity:get_user(user_id=00ebba1e469c4e7bb6302d24c7b89634)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:53.045529 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d84cba9-0081-4cf9-874d-012eac357917 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:53.047432 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 563/1158] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:19:52 2022] GET /identity/v3/users/00ebba1e469c4e7bb6302d24c7b89634 => generated 286 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:53.054683 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785227c2-0e97-491d-adc4-21ca7f988e9c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:53.169428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785227c2-0e97-491d-adc4-21ca7f988e9c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:53.206162 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bb4b96-1aa5-46e6-b12b-bc4b2454ac6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:53.206564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb4b96-1aa5-46e6-b12b-bc4b2454ac6d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:53.206835 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb4b96-1aa5-46e6-b12b-bc4b2454ac6d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:53.207107 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb4b96-1aa5-46e6-b12b-bc4b2454ac6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:53.269251 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:53.282629 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bb4b96-1aa5-46e6-b12b-bc4b2454ac6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:53.284301 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bb4b96-1aa5-46e6-b12b-bc4b2454ac6d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:53.389952 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 596/1159] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:19:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 21:19:53.396838 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d84cba9-0081-4cf9-874d-012eac357917 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:53.504366 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d84cba9-0081-4cf9-874d-012eac357917 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:53.539501 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac3562e-d00a-486a-9bb0-db342a866b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:53.540364 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac3562e-d00a-486a-9bb0-db342a866b5f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:53.540642 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac3562e-d00a-486a-9bb0-db342a866b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:53.540922 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac3562e-d00a-486a-9bb0-db342a866b5f None admin] PATH_INFO: `/v3/projects/db723326dd804ca6a85aeb274243d246` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:53.542112 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac3562e-d00a-486a-9bb0-db342a866b5f None admin] RBAC: Authorizing `identity:get_project(project_id=db723326dd804ca6a85aeb274243d246)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:53.543601 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac3562e-d00a-486a-9bb0-db342a866b5f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:53.545294 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 564/1160] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:19:53 2022] GET /identity/v3/projects/db723326dd804ca6a85aeb274243d246 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:53.551763 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bb4b96-1aa5-46e6-b12b-bc4b2454ac6d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:53.649319 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:53.663674 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bb4b96-1aa5-46e6-b12b-bc4b2454ac6d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:53.701557 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2372caf-68f0-43f9-8e2f-ea3d1254e3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:53.702219 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2372caf-68f0-43f9-8e2f-ea3d1254e3c3 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:53.702532 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2372caf-68f0-43f9-8e2f-ea3d1254e3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:53.702804 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2372caf-68f0-43f9-8e2f-ea3d1254e3c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:53.703640 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2372caf-68f0-43f9-8e2f-ea3d1254e3c3 None admin] RBAC: Adding query filter params (role.id=2b714f2e5c6c469f8fa037bda1ed16f6, scope.project.id=db723326dd804ca6a85aeb274243d246, user.id=00ebba1e469c4e7bb6302d24c7b89634) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:53.703949 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2372caf-68f0-43f9-8e2f-ea3d1254e3c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:53.705748 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2372caf-68f0-43f9-8e2f-ea3d1254e3c3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:53.712554 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 597/1161] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:19:53 2022] GET /identity/v3/role_assignments?user.id=00ebba1e469c4e7bb6302d24c7b89634&scope.project.id=db723326dd804ca6a85aeb274243d246&role.id=2b714f2e5c6c469f8fa037bda1ed16f6 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:56.053306 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31efc296-b348-4cca-8768-5fc84e9f5441 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:56.053922 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31efc296-b348-4cca-8768-5fc84e9f5441 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:56.053922 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31efc296-b348-4cca-8768-5fc84e9f5441 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:56.055300 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 565/1162] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:19:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:19:56.064744 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb247af7-c301-447e-9796-dca0067a4a88 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:56.065493 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb247af7-c301-447e-9796-dca0067a4a88 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:56.065731 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb247af7-c301-447e-9796-dca0067a4a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:56.093185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-bb247af7-c301-447e-9796-dca0067a4a88 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:19:56.095203 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-bb247af7-c301-447e-9796-dca0067a4a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:19:56.210138 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 598/1163] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:19:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:19:56.242419 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31efc296-b348-4cca-8768-5fc84e9f5441 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:56.297056 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:56.384309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31efc296-b348-4cca-8768-5fc84e9f5441 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:56.427408 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c9bb62-c9a5-441c-895c-f739ff12d40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:56.428144 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9bb62-c9a5-441c-895c-f739ff12d40d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:56.428474 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9bb62-c9a5-441c-895c-f739ff12d40d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:56.428812 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9bb62-c9a5-441c-895c-f739ff12d40d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:56.511794 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c9bb62-c9a5-441c-895c-f739ff12d40d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:56.513289 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c9bb62-c9a5-441c-895c-f739ff12d40d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:56.558027 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:56.627139 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 566/1164] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:19:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 21:19:56.637167 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb247af7-c301-447e-9796-dca0067a4a88 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:56.778565 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:56.801417 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb247af7-c301-447e-9796-dca0067a4a88 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:56.840498 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28617067-9101-45d4-b677-1df73d5d98f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:56.841328 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28617067-9101-45d4-b677-1df73d5d98f9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:56.841611 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28617067-9101-45d4-b677-1df73d5d98f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:56.841885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28617067-9101-45d4-b677-1df73d5d98f9 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:56.850342 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28617067-9101-45d4-b677-1df73d5d98f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:56.851815 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28617067-9101-45d4-b677-1df73d5d98f9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:56.868166 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-28617067-9101-45d4-b677-1df73d5d98f9 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 21:19:56.868166 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:19:56.868166 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 19 21:19:56.868166 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 19 21:19:56.868166 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:19:56.868166 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:19:56.868166 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:19:56.868166 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:19:56.868166 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 19 21:19:56.868166 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:19:56.868166 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:19:56.868166 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:19:56.868166 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:19:56.868166 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:19:56.868166 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:19:56.868166 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:19:56.868166 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:19:56.870229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:19:56.870229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:19:56.870229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:19:56.870229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:19:56.870229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:19:56.870229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:19:56.870229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:19:56.870229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:19:56.870229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:19:56.870229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:19:56.870229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:19:56.870229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:19:56.870229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:19:56.870229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:19:56.870229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:19:56.870229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:19:56.872258 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:19:56.872258 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:19:56.872258 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:19:56.872258 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:19:56.872258 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:19:56.872258 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:19:56.872258 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:19:56.872258 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:19:56.872258 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:19:56.872258 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:19:56.872258 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 19 21:19:56.872258 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:19:56.872258 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 21:19:56.872258 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:19:56.872258 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 599/1165] 10.222.0.65 () {56 vars in 1117 bytes} [Thu May 19 21:19:56 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:19:56.878405 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c9bb62-c9a5-441c-895c-f739ff12d40d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:56.988435 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c9bb62-c9a5-441c-895c-f739ff12d40d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:57.026198 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11fe26a-163b-45dd-a8f4-cc619dbd4813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:57.026991 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fe26a-163b-45dd-a8f4-cc619dbd4813 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:57.027263 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fe26a-163b-45dd-a8f4-cc619dbd4813 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:57.027535 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fe26a-163b-45dd-a8f4-cc619dbd4813 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:57.028262 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11fe26a-163b-45dd-a8f4-cc619dbd4813 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:19:57.028630 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11fe26a-163b-45dd-a8f4-cc619dbd4813 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:57.029989 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11fe26a-163b-45dd-a8f4-cc619dbd4813 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:57.038835 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 567/1166] 10.222.0.65 () {56 vars in 1115 bytes} [Thu May 19 21:19:56 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:19:59.367411 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d300fc0c-df32-40df-bc70-a73aab958380 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:59.367988 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d300fc0c-df32-40df-bc70-a73aab958380 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:59.367988 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d300fc0c-df32-40df-bc70-a73aab958380 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:59.369529 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 600/1167] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:19:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:19:59.375469 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a00c2-7a3a-426e-a34d-bd0676761388 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:59.375985 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a00c2-7a3a-426e-a34d-bd0676761388 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:59.376114 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a00c2-7a3a-426e-a34d-bd0676761388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:59.381775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:19:59.395804 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-a37a00c2-7a3a-426e-a34d-bd0676761388 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:19:59.397079 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a37a00c2-7a3a-426e-a34d-bd0676761388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:19:59.471072 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 568/1168] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:19:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:19:59.494430 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d300fc0c-df32-40df-bc70-a73aab958380 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:19:59.615518 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d300fc0c-df32-40df-bc70-a73aab958380 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:19:59.653512 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7a61a-3abe-443b-979b-f47bcdb7d4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:19:59.654088 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7a61a-3abe-443b-979b-f47bcdb7d4ba None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:19:59.654399 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7a61a-3abe-443b-979b-f47bcdb7d4ba None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:19:59.654679 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7a61a-3abe-443b-979b-f47bcdb7d4ba None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:19:59.655449 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc7a61a-3abe-443b-979b-f47bcdb7d4ba None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:19:59.656845 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc7a61a-3abe-443b-979b-f47bcdb7d4ba None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:19:59.675229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 601/1169] 10.222.0.65 () {60 vars in 1138 bytes} [Thu May 19 21:19:59 2022] POST /identity/v3/domains => generated 267 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:20:02.029309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47bca5-a5bf-4c04-b5e2-1729e1f0ec5c None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:20:02.029919 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47bca5-a5bf-4c04-b5e2-1729e1f0ec5c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:20:02.029919 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47bca5-a5bf-4c04-b5e2-1729e1f0ec5c None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:20:02.031369 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 569/1170] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:20:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:20:02.038038 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d9c23-1463-4905-9ffc-099895f61d98 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:20:02.038650 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d9c23-1463-4905-9ffc-099895f61d98 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:20:02.038650 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d9c23-1463-4905-9ffc-099895f61d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:20:02.044204 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:20:02.059106 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-055d9c23-1463-4905-9ffc-099895f61d98 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:20:02.060401 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-055d9c23-1463-4905-9ffc-099895f61d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:20:02.136985 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 602/1171] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:20:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:20:02.161021 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff47bca5-a5bf-4c04-b5e2-1729e1f0ec5c None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:20:02.284202 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff47bca5-a5bf-4c04-b5e2-1729e1f0ec5c None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:20:02.318563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:20:02.324245 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd95b2b-eee8-4c00-b297-e5997300cab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:20:02.325273 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd95b2b-eee8-4c00-b297-e5997300cab6 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:20:02.325645 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd95b2b-eee8-4c00-b297-e5997300cab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:20:02.326005 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd95b2b-eee8-4c00-b297-e5997300cab6 None admin] PATH_INFO: `/v3/domains/d75532a9e40d4e669fe069cbaf0ef31f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:20:02.339003 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd95b2b-eee8-4c00-b297-e5997300cab6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d75532a9e40d4e669fe069cbaf0ef31f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:20:02.340628 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd95b2b-eee8-4c00-b297-e5997300cab6 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:20:02.342445 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 570/1172] 10.222.0.65 () {56 vars in 1183 bytes} [Thu May 19 21:20:02 2022] GET /identity/v3/domains/d75532a9e40d4e669fe069cbaf0ef31f => generated 267 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:20:02.349184 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d9c23-1463-4905-9ffc-099895f61d98 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:20:02.460307 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d9c23-1463-4905-9ffc-099895f61d98 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:20:02.498437 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:20:02.504820 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171a43a-7834-4cc1-b0a4-34d3f9a272e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:20:02.505568 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171a43a-7834-4cc1-b0a4-34d3f9a272e5 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:20:02.505910 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171a43a-7834-4cc1-b0a4-34d3f9a272e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:20:02.506258 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171a43a-7834-4cc1-b0a4-34d3f9a272e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:20:02.507083 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171a43a-7834-4cc1-b0a4-34d3f9a272e5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:20:02.508597 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171a43a-7834-4cc1-b0a4-34d3f9a272e5 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:20:02.538691 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 603/1173] 10.222.0.65 () {60 vars in 1141 bytes} [Thu May 19 21:20:02 2022] POST /identity/v3/projects => generated 361 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:20:04.895239 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013e990-c5a0-4f60-8ac7-4da28f34cbbe None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:20:04.896086 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013e990-c5a0-4f60-8ac7-4da28f34cbbe None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:20:04.896086 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013e990-c5a0-4f60-8ac7-4da28f34cbbe None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:20:04.897631 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 571/1174] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:20:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:20:04.903888 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3758c-a07c-4e51-93ea-f9c078e87721 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:20:04.904493 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3758c-a07c-4e51-93ea-f9c078e87721 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:20:04.904493 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3758c-a07c-4e51-93ea-f9c078e87721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:20:04.923398 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-6ee3758c-a07c-4e51-93ea-f9c078e87721 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:20:04.924732 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6ee3758c-a07c-4e51-93ea-f9c078e87721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:20:05.001530 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 604/1175] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:20:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:20:05.045126 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4013e990-c5a0-4f60-8ac7-4da28f34cbbe None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:20:05.239156 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4013e990-c5a0-4f60-8ac7-4da28f34cbbe None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:20:05.303615 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8954edd5-e2d1-459c-8bae-b927717ca2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:20:05.305090 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8954edd5-e2d1-459c-8bae-b927717ca2e1 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:20:05.305577 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8954edd5-e2d1-459c-8bae-b927717ca2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:20:05.306073 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8954edd5-e2d1-459c-8bae-b927717ca2e1 None admin] PATH_INFO: `/v3/domains/d75532a9e40d4e669fe069cbaf0ef31f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:20:05.308330 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8954edd5-e2d1-459c-8bae-b927717ca2e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d75532a9e40d4e669fe069cbaf0ef31f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:20:05.311044 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8954edd5-e2d1-459c-8bae-b927717ca2e1 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:20:05.314063 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 572/1176] 10.222.0.65 () {56 vars in 1183 bytes} [Thu May 19 21:20:05 2022] GET /identity/v3/domains/d75532a9e40d4e669fe069cbaf0ef31f => generated 267 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:20:05.323431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee3758c-a07c-4e51-93ea-f9c078e87721 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:20:05.435932 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee3758c-a07c-4e51-93ea-f9c078e87721 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:20:05.451274 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:20:05.478569 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f40844-fa9b-4acf-a735-e78ecf8a9d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:20:05.479644 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f40844-fa9b-4acf-a735-e78ecf8a9d57 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:20:05.480085 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f40844-fa9b-4acf-a735-e78ecf8a9d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:20:05.480407 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f40844-fa9b-4acf-a735-e78ecf8a9d57 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:20:05.481241 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f40844-fa9b-4acf-a735-e78ecf8a9d57 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:20:05.482730 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f40844-fa9b-4acf-a735-e78ecf8a9d57 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:20:05.506166 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 605/1177] 10.222.0.65 () {60 vars in 1132 bytes} [Thu May 19 21:20:05 2022] POST /identity/v3/users => generated 311 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:20:07.914447 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712c50d-e95e-4d87-af7a-020cd679bf37 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:20:07.915066 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712c50d-e95e-4d87-af7a-020cd679bf37 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:20:07.915066 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712c50d-e95e-4d87-af7a-020cd679bf37 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:20:07.916370 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 573/1178] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:20:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:20:07.922372 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78012f6-fd52-4214-a8d8-4a4d6b90e9c8 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:20:07.923146 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78012f6-fd52-4214-a8d8-4a4d6b90e9c8 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:20:07.923325 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78012f6-fd52-4214-a8d8-4a4d6b90e9c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:20:07.942595 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a78012f6-fd52-4214-a8d8-4a4d6b90e9c8 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:20:07.943822 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a78012f6-fd52-4214-a8d8-4a4d6b90e9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:20:08.026529 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 606/1179] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:20:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:20:08.052035 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8712c50d-e95e-4d87-af7a-020cd679bf37 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:20:08.121990 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:20:08.169133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8712c50d-e95e-4d87-af7a-020cd679bf37 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:20:08.206892 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745065fa-e872-4d78-9b39-6337dbb32dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:20:08.207713 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745065fa-e872-4d78-9b39-6337dbb32dae None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:20:08.207995 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745065fa-e872-4d78-9b39-6337dbb32dae None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:20:08.208268 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745065fa-e872-4d78-9b39-6337dbb32dae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:20:08.220919 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745065fa-e872-4d78-9b39-6337dbb32dae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:20:08.228960 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-745065fa-e872-4d78-9b39-6337dbb32dae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:20:08.228960 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:20:08.228960 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:20:08.228960 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:20:08.228960 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:20:08.228960 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:20:08.228960 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:20:08.228960 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:20:08.228960 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:20:08.228960 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:20:08.228960 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:20:08.228960 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:20:08.228960 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:20:08.228960 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:20:08.228960 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:20:08.228960 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:20:08.230745 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:20:08.230745 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:20:08.230745 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:20:08.230745 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:20:08.230745 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:20:08.230745 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:20:08.230745 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:20:08.230745 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:20:08.230745 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:20:08.230745 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:20:08.230745 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:20:08.230745 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:20:08.230745 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:20:08.230745 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:20:08.230745 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:20:08.230745 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:20:08.231799 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:20:08.231799 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:20:08.231799 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:20:08.231799 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:20:08.231799 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:20:08.231799 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:20:08.231799 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:20:08.231799 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:20:08.231799 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:20:08.231799 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:20:08.231799 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:20:08.231799 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:20:08.231799 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 574/1180] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 21:20:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:20:08.237843 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78012f6-fd52-4214-a8d8-4a4d6b90e9c8 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:20:08.344255 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:20:08.348061 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78012f6-fd52-4214-a8d8-4a4d6b90e9c8 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:20:08.387685 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b006b66f-0dfe-4ee4-ae3f-e1ab0b7b2e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:20:08.388272 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b006b66f-0dfe-4ee4-ae3f-e1ab0b7b2e5a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:20:08.388553 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b006b66f-0dfe-4ee4-ae3f-e1ab0b7b2e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:20:08.388825 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b006b66f-0dfe-4ee4-ae3f-e1ab0b7b2e5a None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:20:08.389584 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b006b66f-0dfe-4ee4-ae3f-e1ab0b7b2e5a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:20:08.389886 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b006b66f-0dfe-4ee4-ae3f-e1ab0b7b2e5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:20:08.391328 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b006b66f-0dfe-4ee4-ae3f-e1ab0b7b2e5a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:20:08.401066 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 607/1181] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:20:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:20:08.407255 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745065fa-e872-4d78-9b39-6337dbb32dae None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:20:08.540171 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745065fa-e872-4d78-9b39-6337dbb32dae None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:20:08.565191 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:20:08.581921 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebcb9d4-bb80-4c2f-adbf-019d7d470ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:20:08.582897 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebcb9d4-bb80-4c2f-adbf-019d7d470ebb None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:20:08.583215 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebcb9d4-bb80-4c2f-adbf-019d7d470ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:20:08.583492 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebcb9d4-bb80-4c2f-adbf-019d7d470ebb None admin] PATH_INFO: `/v3/users/85f9df00529d4e74a74d842510d1145d` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:20:08.600798 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebcb9d4-bb80-4c2f-adbf-019d7d470ebb None admin] RBAC: Authorizing `identity:get_user(user_id=85f9df00529d4e74a74d842510d1145d)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:20:08.602367 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebcb9d4-bb80-4c2f-adbf-019d7d470ebb None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:20:08.604246 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 575/1182] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:20:08 2022] GET /identity/v3/users/85f9df00529d4e74a74d842510d1145d => generated 311 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:20:08.611735 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b006b66f-0dfe-4ee4-ae3f-e1ab0b7b2e5a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:20:08.734141 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b006b66f-0dfe-4ee4-ae3f-e1ab0b7b2e5a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:20:08.776863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76403891-5be4-4f5d-87ab-60f7ae604d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:20:08.777603 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76403891-5be4-4f5d-87ab-60f7ae604d74 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:20:08.777944 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76403891-5be4-4f5d-87ab-60f7ae604d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:20:08.778290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76403891-5be4-4f5d-87ab-60f7ae604d74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:20:08.811317 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:20:08.879544 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76403891-5be4-4f5d-87ab-60f7ae604d74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:20:08.882229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76403891-5be4-4f5d-87ab-60f7ae604d74 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:20:09.038898 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 608/1183] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:20:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 21:20:09.048017 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebcb9d4-bb80-4c2f-adbf-019d7d470ebb None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:20:09.166566 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebcb9d4-bb80-4c2f-adbf-019d7d470ebb None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:20:09.204516 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5925d4-bc4e-4e32-ab79-4f791ca5f615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:20:09.205397 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5925d4-bc4e-4e32-ab79-4f791ca5f615 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:20:09.205684 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5925d4-bc4e-4e32-ab79-4f791ca5f615 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:20:09.205956 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5925d4-bc4e-4e32-ab79-4f791ca5f615 None admin] PATH_INFO: `/v3/projects/45b6ebbb39df47d79760594a1479c1d0` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:20:09.207252 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5925d4-bc4e-4e32-ab79-4f791ca5f615 None admin] RBAC: Authorizing `identity:get_project(project_id=45b6ebbb39df47d79760594a1479c1d0)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:20:09.208704 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5925d4-bc4e-4e32-ab79-4f791ca5f615 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:20:09.210458 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 576/1184] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:20:09 2022] GET /identity/v3/projects/45b6ebbb39df47d79760594a1479c1d0 => generated 361 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:20:09.220887 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76403891-5be4-4f5d-87ab-60f7ae604d74 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:20:09.329186 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:20:09.401015 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76403891-5be4-4f5d-87ab-60f7ae604d74 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:20:09.468957 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4e9a1f-643f-4c31-bc53-f8baccc65be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:20:09.471101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e9a1f-643f-4c31-bc53-f8baccc65be5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:20:09.471272 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e9a1f-643f-4c31-bc53-f8baccc65be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:20:09.471689 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e9a1f-643f-4c31-bc53-f8baccc65be5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:20:09.473349 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4e9a1f-643f-4c31-bc53-f8baccc65be5 None admin] RBAC: Adding query filter params (role.id=2b714f2e5c6c469f8fa037bda1ed16f6, scope.project.id=45b6ebbb39df47d79760594a1479c1d0, user.id=85f9df00529d4e74a74d842510d1145d) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:20:09.473923 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4e9a1f-643f-4c31-bc53-f8baccc65be5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:20:09.476721 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4e9a1f-643f-4c31-bc53-f8baccc65be5 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:20:09.488755 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 609/1185] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:20:09 2022] GET /identity/v3/role_assignments?user.id=85f9df00529d4e74a74d842510d1145d&scope.project.id=45b6ebbb39df47d79760594a1479c1d0&role.id=2b714f2e5c6c469f8fa037bda1ed16f6 => generated 259 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:20:11.927109 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b952dc3-fe9c-4fce-a9aa-5ba673927da5 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:20:11.928001 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b952dc3-fe9c-4fce-a9aa-5ba673927da5 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:20:11.928001 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b952dc3-fe9c-4fce-a9aa-5ba673927da5 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:20:11.929651 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 577/1186] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:20:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:20:11.936045 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61054a3-4539-44f9-b9c5-ca145bfcf712 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:20:11.936650 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61054a3-4539-44f9-b9c5-ca145bfcf712 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:20:11.936650 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61054a3-4539-44f9-b9c5-ca145bfcf712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:20:11.954818 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e61054a3-4539-44f9-b9c5-ca145bfcf712 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:20:11.956245 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e61054a3-4539-44f9-b9c5-ca145bfcf712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:20:12.039286 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 610/1187] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:20:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:20:12.061962 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b952dc3-fe9c-4fce-a9aa-5ba673927da5 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:20:12.117590 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:20:12.177974 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b952dc3-fe9c-4fce-a9aa-5ba673927da5 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:20:12.215339 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa698036-c759-4d4e-b5e9-1766a8a62abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:20:12.216148 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa698036-c759-4d4e-b5e9-1766a8a62abf None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:20:12.216431 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa698036-c759-4d4e-b5e9-1766a8a62abf None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:20:12.216703 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa698036-c759-4d4e-b5e9-1766a8a62abf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:20:12.227898 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa698036-c759-4d4e-b5e9-1766a8a62abf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:20:12.235561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-aa698036-c759-4d4e-b5e9-1766a8a62abf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:20:12.235561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:20:12.235561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:20:12.235561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:20:12.235561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:20:12.235561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:20:12.235561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:20:12.235561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:20:12.235561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:20:12.235561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:20:12.235561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:20:12.235561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:20:12.235561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:20:12.235561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:20:12.235561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:20:12.235561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:20:12.237387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:20:12.237387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:20:12.237387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:20:12.237387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:20:12.237387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:20:12.237387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:20:12.237387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:20:12.237387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:20:12.237387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:20:12.237387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:20:12.237387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:20:12.237387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:20:12.237387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:20:12.237387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:20:12.237387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:20:12.237387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:20:12.239133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:20:12.239133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:20:12.239133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:20:12.239133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:20:12.239133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:20:12.239133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:20:12.239133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:20:12.239133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:20:12.239133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:20:12.239133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:20:12.239133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:20:12.239133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:20:12.239133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 578/1188] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 21:20:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:20:12.243425 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61054a3-4539-44f9-b9c5-ca145bfcf712 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:20:12.309046 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:20:12.364517 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61054a3-4539-44f9-b9c5-ca145bfcf712 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:20:12.405578 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a62b9-de6b-443b-be27-e74f123702d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:20:12.406174 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a62b9-de6b-443b-be27-e74f123702d7 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:20:12.406479 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a62b9-de6b-443b-be27-e74f123702d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:20:12.406755 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a62b9-de6b-443b-be27-e74f123702d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:20:12.407618 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a62b9-de6b-443b-be27-e74f123702d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:20:12.408049 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a62b9-de6b-443b-be27-e74f123702d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:20:12.409854 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a62b9-de6b-443b-be27-e74f123702d7 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:20:12.419718 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 611/1189] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:20:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:20:12.427138 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa698036-c759-4d4e-b5e9-1766a8a62abf None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:20:12.541933 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa698036-c759-4d4e-b5e9-1766a8a62abf None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:20:12.549791 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:20:12.580526 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83158190-b9bb-4870-87b6-de013bfa4c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:20:12.581420 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83158190-b9bb-4870-87b6-de013bfa4c49 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:20:12.581726 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83158190-b9bb-4870-87b6-de013bfa4c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:20:12.582007 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83158190-b9bb-4870-87b6-de013bfa4c49 None admin] PATH_INFO: `/v3/users/85f9df00529d4e74a74d842510d1145d` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:20:12.583332 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83158190-b9bb-4870-87b6-de013bfa4c49 None admin] RBAC: Authorizing `identity:get_user(user_id=85f9df00529d4e74a74d842510d1145d)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:20:12.584821 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83158190-b9bb-4870-87b6-de013bfa4c49 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:20:12.586541 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 579/1190] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:20:12 2022] GET /identity/v3/users/85f9df00529d4e74a74d842510d1145d => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:20:12.592551 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a62b9-de6b-443b-be27-e74f123702d7 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:20:12.698662 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a62b9-de6b-443b-be27-e74f123702d7 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:20:12.716616 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:20:12.734428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b36f6-b996-4e09-bc57-63b21b84a688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:20:12.735303 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b36f6-b996-4e09-bc57-63b21b84a688 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:20:12.735814 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b36f6-b996-4e09-bc57-63b21b84a688 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:20:12.736269 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b36f6-b996-4e09-bc57-63b21b84a688 None admin] PATH_INFO: `/v3/projects/45b6ebbb39df47d79760594a1479c1d0` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:20:12.737435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090b36f6-b996-4e09-bc57-63b21b84a688 None admin] RBAC: Authorizing `identity:get_project(project_id=45b6ebbb39df47d79760594a1479c1d0)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:20:12.738907 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090b36f6-b996-4e09-bc57-63b21b84a688 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:20:12.740644 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 612/1191] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:20:12 2022] GET /identity/v3/projects/45b6ebbb39df47d79760594a1479c1d0 => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:20:12.746822 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83158190-b9bb-4870-87b6-de013bfa4c49 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:20:12.852761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83158190-b9bb-4870-87b6-de013bfa4c49 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:20:12.890276 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c821eed-9a3a-4470-b86b-e0ec4ac3b044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:20:12.891176 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c821eed-9a3a-4470-b86b-e0ec4ac3b044 None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:20:12.891451 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c821eed-9a3a-4470-b86b-e0ec4ac3b044 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:20:12.891699 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c821eed-9a3a-4470-b86b-e0ec4ac3b044 None admin] PATH_INFO: `/v3/projects/45b6ebbb39df47d79760594a1479c1d0/users/85f9df00529d4e74a74d842510d1145d/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:20:12.893853 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c821eed-9a3a-4470-b86b-e0ec4ac3b044 None admin] RBAC: Authorizing `identity:create_grant(project_id=45b6ebbb39df47d79760594a1479c1d0, user_id=85f9df00529d4e74a74d842510d1145d, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:20:12.895346 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c821eed-9a3a-4470-b86b-e0ec4ac3b044 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:20:12.910619 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 580/1192] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 21:20:12 2022] PUT /identity/v3/projects/45b6ebbb39df47d79760594a1479c1d0/users/85f9df00529d4e74a74d842510d1145d/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:20:15.134438 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fcb7d-1cf0-4b7f-99e2-a035aed02fd3 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:20:15.135024 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fcb7d-1cf0-4b7f-99e2-a035aed02fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:20:15.135024 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fcb7d-1cf0-4b7f-99e2-a035aed02fd3 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:20:15.136215 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 613/1193] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:20:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:20:15.141976 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a026b6-b8bf-4105-a7ae-11c32a812edb None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:20:15.142490 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a026b6-b8bf-4105-a7ae-11c32a812edb None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:20:15.142490 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a026b6-b8bf-4105-a7ae-11c32a812edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:20:15.161225 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-06a026b6-b8bf-4105-a7ae-11c32a812edb None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:20:15.162471 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-06a026b6-b8bf-4105-a7ae-11c32a812edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:20:15.193132 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:20:15.240881 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 581/1194] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:20:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:20:15.264381 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569fcb7d-1cf0-4b7f-99e2-a035aed02fd3 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:20:15.382808 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569fcb7d-1cf0-4b7f-99e2-a035aed02fd3 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:20:15.441365 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df47443-49d5-41da-9e0b-d2a0491580dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:20:15.442828 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df47443-49d5-41da-9e0b-d2a0491580dc None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:20:15.443335 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df47443-49d5-41da-9e0b-d2a0491580dc None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:20:15.443831 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df47443-49d5-41da-9e0b-d2a0491580dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:20:15.463756 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df47443-49d5-41da-9e0b-d2a0491580dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:20:15.478911 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-0df47443-49d5-41da-9e0b-d2a0491580dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:20:15.478911 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:20:15.478911 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:20:15.478911 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:20:15.478911 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:20:15.478911 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:20:15.478911 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:20:15.478911 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:20:15.478911 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:20:15.478911 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:20:15.478911 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:20:15.478911 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:20:15.478911 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:20:15.478911 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:20:15.478911 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:20:15.478911 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:20:15.480038 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:20:15.480038 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:20:15.480038 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:20:15.480038 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:20:15.480038 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:20:15.480038 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:20:15.480038 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:20:15.480038 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:20:15.480038 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:20:15.480038 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:20:15.480038 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:20:15.480038 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:20:15.480038 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:20:15.480038 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:20:15.480038 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:20:15.480038 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:20:15.481085 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:20:15.481085 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:20:15.481085 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:20:15.481085 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:20:15.481085 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:20:15.481085 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:20:15.481085 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:20:15.481085 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:20:15.481085 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:20:15.481085 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:20:15.481085 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:20:15.481085 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:20:15.481085 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 614/1195] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 21:20:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:20:15.488093 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a026b6-b8bf-4105-a7ae-11c32a812edb None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:20:15.604899 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a026b6-b8bf-4105-a7ae-11c32a812edb None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:20:15.647259 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f942654c-ae5c-4176-9081-e284100da5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:20:15.647961 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942654c-ae5c-4176-9081-e284100da5b1 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:20:15.648251 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942654c-ae5c-4176-9081-e284100da5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:20:15.648527 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942654c-ae5c-4176-9081-e284100da5b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:20:15.649318 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f942654c-ae5c-4176-9081-e284100da5b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:20:15.649627 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f942654c-ae5c-4176-9081-e284100da5b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:20:15.651127 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f942654c-ae5c-4176-9081-e284100da5b1 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:20:15.661676 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 582/1196] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:20:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:20:15.668339 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df47443-49d5-41da-9e0b-d2a0491580dc None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:20:15.725266 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:20:15.788131 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df47443-49d5-41da-9e0b-d2a0491580dc None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:20:15.825776 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6be594-f09f-4802-963b-70c5e09e2c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:20:15.826690 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6be594-f09f-4802-963b-70c5e09e2c39 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:20:15.826971 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6be594-f09f-4802-963b-70c5e09e2c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:20:15.827244 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6be594-f09f-4802-963b-70c5e09e2c39 None admin] PATH_INFO: `/v3/users/85f9df00529d4e74a74d842510d1145d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:20:15.828487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6be594-f09f-4802-963b-70c5e09e2c39 None admin] RBAC: Authorizing `identity:get_user(user_id=85f9df00529d4e74a74d842510d1145d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:20:15.830381 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6be594-f09f-4802-963b-70c5e09e2c39 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:20:15.832051 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 615/1197] 10.222.0.65 () {56 vars in 1177 bytes} [Thu May 19 21:20:15 2022] GET /identity/v3/users/85f9df00529d4e74a74d842510d1145d => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:20:15.838066 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f942654c-ae5c-4176-9081-e284100da5b1 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:20:15.873209 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:20:15.953482 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f942654c-ae5c-4176-9081-e284100da5b1 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:20:15.990362 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e13a1f-3b82-4922-a24e-b05346df10bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:20:15.990942 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e13a1f-3b82-4922-a24e-b05346df10bd None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:20:15.991224 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e13a1f-3b82-4922-a24e-b05346df10bd None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:20:15.991496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e13a1f-3b82-4922-a24e-b05346df10bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:20:16.072308 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e13a1f-3b82-4922-a24e-b05346df10bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:20:16.073789 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e13a1f-3b82-4922-a24e-b05346df10bd None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:20:16.102413 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:20:16.181999 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 583/1198] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:20:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 21:20:16.188510 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6be594-f09f-4802-963b-70c5e09e2c39 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:20:16.305281 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6be594-f09f-4802-963b-70c5e09e2c39 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:20:16.320306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:20:16.343622 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1254ee5d-78e0-493f-94e6-2442f9a905df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:20:16.344512 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254ee5d-78e0-493f-94e6-2442f9a905df None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:20:16.344954 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254ee5d-78e0-493f-94e6-2442f9a905df None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:20:16.345902 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254ee5d-78e0-493f-94e6-2442f9a905df None admin] PATH_INFO: `/v3/projects/45b6ebbb39df47d79760594a1479c1d0` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:20:16.347241 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1254ee5d-78e0-493f-94e6-2442f9a905df None admin] RBAC: Authorizing `identity:get_project(project_id=45b6ebbb39df47d79760594a1479c1d0)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:20:16.348693 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1254ee5d-78e0-493f-94e6-2442f9a905df None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:20:16.350477 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 616/1199] 10.222.0.65 () {56 vars in 1186 bytes} [Thu May 19 21:20:16 2022] GET /identity/v3/projects/45b6ebbb39df47d79760594a1479c1d0 => generated 361 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:20:16.356859 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e13a1f-3b82-4922-a24e-b05346df10bd None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:20:16.472634 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e13a1f-3b82-4922-a24e-b05346df10bd None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:20:16.499208 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:20:16.509519 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b33e2-02ad-4aaf-a4bd-635a071656a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:20:16.510188 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b33e2-02ad-4aaf-a4bd-635a071656a2 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:20:16.510498 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b33e2-02ad-4aaf-a4bd-635a071656a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:20:16.510768 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b33e2-02ad-4aaf-a4bd-635a071656a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:20:16.511595 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4b33e2-02ad-4aaf-a4bd-635a071656a2 None admin] RBAC: Adding query filter params (role.id=2b714f2e5c6c469f8fa037bda1ed16f6, scope.project.id=45b6ebbb39df47d79760594a1479c1d0, user.id=85f9df00529d4e74a74d842510d1145d) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:20:16.511902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4b33e2-02ad-4aaf-a4bd-635a071656a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:20:16.513378 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4b33e2-02ad-4aaf-a4bd-635a071656a2 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:20:16.520120 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 584/1200] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:20:16 2022] GET /identity/v3/role_assignments?user.id=85f9df00529d4e74a74d842510d1145d&scope.project.id=45b6ebbb39df47d79760594a1479c1d0&role.id=2b714f2e5c6c469f8fa037bda1ed16f6 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:20:26.339152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=9 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:20:26.516976 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=5 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:19.100506 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f7683-6246-49f6-94ea-1714c907a858 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:19.100506 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f7683-6246-49f6-94ea-1714c907a858 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:19.100506 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f7683-6246-49f6-94ea-1714c907a858 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:19.100506 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 617/1201] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:21:19.109506 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4ad10-9d95-4a42-a3af-427db6e927c7 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:19.109506 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4ad10-9d95-4a42-a3af-427db6e927c7 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:19.109506 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4ad10-9d95-4a42-a3af-427db6e927c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:19.131732 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-93b4ad10-9d95-4a42-a3af-427db6e927c7 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:21:19.133249 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-93b4ad10-9d95-4a42-a3af-427db6e927c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:21:19.237567 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 585/1202] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:21:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:21:19.268350 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f7683-6246-49f6-94ea-1714c907a858 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:19.448502 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f7683-6246-49f6-94ea-1714c907a858 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:19.498718 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d836efdd-e960-4885-8e4e-8515383b8057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:19.499767 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836efdd-e960-4885-8e4e-8515383b8057 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:19.500050 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836efdd-e960-4885-8e4e-8515383b8057 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:19.500442 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836efdd-e960-4885-8e4e-8515383b8057 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:19.514898 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d836efdd-e960-4885-8e4e-8515383b8057 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:19.517038 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d836efdd-e960-4885-8e4e-8515383b8057 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:19.546259 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-d836efdd-e960-4885-8e4e-8515383b8057 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:21:19.546259 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:21:19.546259 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:21:19.546259 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:21:19.546259 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:21:19.546259 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:21:19.546259 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:21:19.546259 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:21:19.546259 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:21:19.546259 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:21:19.546259 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:21:19.546259 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:21:19.546259 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:21:19.546259 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:21:19.546259 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:21:19.547487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:21:19.547487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:21:19.547487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:21:19.547487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:21:19.547487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:21:19.547487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:21:19.547487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:21:19.547487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:21:19.547487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:21:19.547487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:21:19.547487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:21:19.547487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:21:19.547487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:21:19.547487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:21:19.547487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:21:19.547487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:21:19.547487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:21:19.548590 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:21:19.548590 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:21:19.548590 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:21:19.548590 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:21:19.549199 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 618/1203] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:21:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:21:19.556267 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b4ad10-9d95-4a42-a3af-427db6e927c7 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:19.716491 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b4ad10-9d95-4a42-a3af-427db6e927c7 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:19.768677 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:19.778064 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b101190e-3ac4-4472-ac33-d884c07a1e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:19.779251 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101190e-3ac4-4472-ac33-d884c07a1e03 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:19.779578 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101190e-3ac4-4472-ac33-d884c07a1e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:19.779920 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101190e-3ac4-4472-ac33-d884c07a1e03 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:19.780881 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b101190e-3ac4-4472-ac33-d884c07a1e03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:21:19.781375 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b101190e-3ac4-4472-ac33-d884c07a1e03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:19.786224 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b101190e-3ac4-4472-ac33-d884c07a1e03 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:19.800574 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 586/1204] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:21:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:21:19.807711 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d836efdd-e960-4885-8e4e-8515383b8057 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:19.897424 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:19.962525 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d836efdd-e960-4885-8e4e-8515383b8057 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:20.017343 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45516126-f8d1-42ca-96c9-8bf03a4420f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:20.018101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45516126-f8d1-42ca-96c9-8bf03a4420f3 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:20.018408 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45516126-f8d1-42ca-96c9-8bf03a4420f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:20.018717 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45516126-f8d1-42ca-96c9-8bf03a4420f3 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:20.019663 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45516126-f8d1-42ca-96c9-8bf03a4420f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:20.021531 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45516126-f8d1-42ca-96c9-8bf03a4420f3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:20.053880 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 619/1205] 10.222.0.65 () {60 vars in 1132 bytes} [Thu May 19 21:21:19 2022] POST /identity/v3/users => generated 251 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:21:22.592461 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b97f5d-6648-493a-bacd-3d0d834c7941 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:22.593173 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b97f5d-6648-493a-bacd-3d0d834c7941 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:22.593173 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b97f5d-6648-493a-bacd-3d0d834c7941 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:22.594466 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 587/1206] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:21:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:21:22.601876 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f46dd-65a2-4252-aea7-a9d23748208c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:22.602523 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f46dd-65a2-4252-aea7-a9d23748208c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:22.602625 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f46dd-65a2-4252-aea7-a9d23748208c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:22.624572 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-293f46dd-65a2-4252-aea7-a9d23748208c None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:21:22.625865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-293f46dd-65a2-4252-aea7-a9d23748208c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:21:22.737396 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 620/1207] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:21:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:21:22.762433 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b97f5d-6648-493a-bacd-3d0d834c7941 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:22.897957 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b97f5d-6648-493a-bacd-3d0d834c7941 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:22.940870 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9eec87-25a8-4fd8-848f-464797ea5b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:22.941648 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9eec87-25a8-4fd8-848f-464797ea5b16 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:22.941937 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9eec87-25a8-4fd8-848f-464797ea5b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:22.942307 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9eec87-25a8-4fd8-848f-464797ea5b16 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:22.955642 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9eec87-25a8-4fd8-848f-464797ea5b16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:21:22.972815 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-0d9eec87-25a8-4fd8-848f-464797ea5b16 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:21:22.972815 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:21:22.972815 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:21:22.972815 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:21:22.972815 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:21:22.972815 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:21:22.972815 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:21:22.972815 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:21:22.972815 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:21:22.972815 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:21:22.972815 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:21:22.972815 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:21:22.972815 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:21:22.972815 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:21:22.972815 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:21:22.972815 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:21:22.973906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:21:22.973906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:21:22.973906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:21:22.973906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:21:22.973906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:21:22.973906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:21:22.973906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:21:22.973906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:21:22.973906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:21:22.973906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:21:22.973906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:21:22.973906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:21:22.973906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:21:22.973906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:21:22.973906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:21:22.973906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:21:22.975124 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:21:22.975124 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:21:22.975124 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:21:22.975124 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:21:22.975124 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:21:22.975124 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:21:22.975124 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:21:22.975124 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:21:22.975124 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:21:22.975124 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:21:22.975124 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:21:22.975124 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:21:22.975870 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 588/1208] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 21:21:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:21:22.985585 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f46dd-65a2-4252-aea7-a9d23748208c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:23.030012 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:23.113000 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f46dd-65a2-4252-aea7-a9d23748208c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:23.164486 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5088f765-b24a-47d1-b04b-4846296ed452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:23.165493 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088f765-b24a-47d1-b04b-4846296ed452 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:23.165908 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088f765-b24a-47d1-b04b-4846296ed452 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:23.166312 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088f765-b24a-47d1-b04b-4846296ed452 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:23.167364 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5088f765-b24a-47d1-b04b-4846296ed452 None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:21:23.167963 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5088f765-b24a-47d1-b04b-4846296ed452 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:23.170497 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5088f765-b24a-47d1-b04b-4846296ed452 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:23.183910 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 621/1209] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 21:21:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:21:23.190176 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9eec87-25a8-4fd8-848f-464797ea5b16 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:23.254616 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:23.318818 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9eec87-25a8-4fd8-848f-464797ea5b16 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:23.370394 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b618d732-4d5d-43cd-b45f-e2d4675c5c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:23.371175 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618d732-4d5d-43cd-b45f-e2d4675c5c42 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:23.371601 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618d732-4d5d-43cd-b45f-e2d4675c5c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:23.371958 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618d732-4d5d-43cd-b45f-e2d4675c5c42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:23.387076 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b618d732-4d5d-43cd-b45f-e2d4675c5c42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:23.388625 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b618d732-4d5d-43cd-b45f-e2d4675c5c42 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:23.425481 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-b618d732-4d5d-43cd-b45f-e2d4675c5c42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:21:23.425481 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:21:23.425481 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:21:23.425481 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:21:23.425481 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:21:23.425481 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:21:23.425481 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:21:23.425481 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:21:23.425481 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:21:23.425481 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:21:23.425481 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:21:23.425481 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:21:23.425481 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:21:23.425481 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:21:23.425481 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:21:23.427546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:21:23.427546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:21:23.427546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:21:23.427546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:21:23.427546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:21:23.427546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:21:23.427546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:21:23.427546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:21:23.427546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:21:23.427546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:21:23.427546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:21:23.427546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:21:23.427546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:21:23.427546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:21:23.427546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:21:23.427546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:21:23.427546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:21:23.429545 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:21:23.429545 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:21:23.429545 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:21:23.429545 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:21:23.429545 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 589/1210] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:21:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:21:23.434590 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5088f765-b24a-47d1-b04b-4846296ed452 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:23.569163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:23.572592 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5088f765-b24a-47d1-b04b-4846296ed452 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:23.618208 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94504f1-82d3-441e-9352-a27414d78fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:23.619856 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94504f1-82d3-441e-9352-a27414d78fcd None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:23.621239 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94504f1-82d3-441e-9352-a27414d78fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:23.621239 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94504f1-82d3-441e-9352-a27414d78fcd None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:23.621536 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94504f1-82d3-441e-9352-a27414d78fcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:21:23.621855 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94504f1-82d3-441e-9352-a27414d78fcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:23.624287 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94504f1-82d3-441e-9352-a27414d78fcd None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:23.640193 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 622/1211] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:21:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:21:23.646287 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b618d732-4d5d-43cd-b45f-e2d4675c5c42 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:23.766720 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b618d732-4d5d-43cd-b45f-e2d4675c5c42 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:23.769137 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:23.818339 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7dc483-4e80-45d2-a933-fa26f7871f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:23.819277 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7dc483-4e80-45d2-a933-fa26f7871f78 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:23.819587 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7dc483-4e80-45d2-a933-fa26f7871f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:23.819982 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7dc483-4e80-45d2-a933-fa26f7871f78 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:23.829242 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7dc483-4e80-45d2-a933-fa26f7871f78 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:23.830952 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7dc483-4e80-45d2-a933-fa26f7871f78 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:23.849108 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-ca7dc483-4e80-45d2-a933-fa26f7871f78 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 21:21:23.849108 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:21:23.849108 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:21:23.849108 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:21:23.849108 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:21:23.849108 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:21:23.849108 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:21:23.849108 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:21:23.849108 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:21:23.849108 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:21:23.849108 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:21:23.849108 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:21:23.849108 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:21:23.849108 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:21:23.849108 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:21:23.849108 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:21:23.850219 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:21:23.850219 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:21:23.850219 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:21:23.850219 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:21:23.850219 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:21:23.850219 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:21:23.850219 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:21:23.850219 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:21:23.850219 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:21:23.850219 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:21:23.850219 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:21:23.850219 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:21:23.850219 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:21:23.850219 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:21:23.850219 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:21:23.850219 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:21:23.850219 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:21:23.850219 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:21:23.851287 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:21:23.851287 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:21:23.851287 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:21:23.851287 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:21:23.851287 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:21:23.851287 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:21:23.851287 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:21:23.851287 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:21:23.851287 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 21:21:23.851287 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:21:23.851287 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 590/1212] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:21:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:21:23.856836 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94504f1-82d3-441e-9352-a27414d78fcd None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:23.991788 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94504f1-82d3-441e-9352-a27414d78fcd None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:24.043874 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3effbc-1d33-4a5b-97d8-d43740314505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:24.044608 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3effbc-1d33-4a5b-97d8-d43740314505 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:24.044940 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3effbc-1d33-4a5b-97d8-d43740314505 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:24.045308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3effbc-1d33-4a5b-97d8-d43740314505 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:24.046126 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3effbc-1d33-4a5b-97d8-d43740314505 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:21:24.046497 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3effbc-1d33-4a5b-97d8-d43740314505 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:24.047965 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3effbc-1d33-4a5b-97d8-d43740314505 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:24.071017 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 623/1213] 10.222.0.65 () {56 vars in 1143 bytes} [Thu May 19 21:21:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:21:24.077047 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7dc483-4e80-45d2-a933-fa26f7871f78 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:24.220330 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7dc483-4e80-45d2-a933-fa26f7871f78 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:24.267399 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26c5271-631f-42d3-a6eb-47bf9c4b3276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:24.268020 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c5271-631f-42d3-a6eb-47bf9c4b3276 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:24.268330 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c5271-631f-42d3-a6eb-47bf9c4b3276 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:24.268615 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c5271-631f-42d3-a6eb-47bf9c4b3276 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:24.315617 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:24.374627 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26c5271-631f-42d3-a6eb-47bf9c4b3276 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:24.377336 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26c5271-631f-42d3-a6eb-47bf9c4b3276 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:24.517522 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 591/1214] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:21:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 21:21:24.523949 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3effbc-1d33-4a5b-97d8-d43740314505 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:24.548242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:24.667449 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3effbc-1d33-4a5b-97d8-d43740314505 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:24.721971 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f85855-2035-46de-9b9a-73fdf1101061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:24.723309 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f85855-2035-46de-9b9a-73fdf1101061 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:24.723882 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f85855-2035-46de-9b9a-73fdf1101061 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:24.724376 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f85855-2035-46de-9b9a-73fdf1101061 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:24.739197 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f85855-2035-46de-9b9a-73fdf1101061 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:24.741321 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f85855-2035-46de-9b9a-73fdf1101061 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:24.765418 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-54f85855-2035-46de-9b9a-73fdf1101061 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:21:24.765418 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:21:24.765418 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:21:24.765418 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:21:24.765418 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:21:24.765418 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:21:24.765418 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:21:24.765418 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:21:24.765418 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:21:24.765418 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:21:24.765418 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:21:24.765418 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:21:24.765418 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:21:24.765418 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:21:24.765418 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:21:24.766496 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:21:24.766496 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:21:24.766496 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:21:24.766496 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:21:24.766496 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:21:24.766496 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:21:24.766496 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:21:24.766496 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:21:24.766496 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:21:24.766496 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:21:24.766496 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:21:24.766496 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:21:24.766496 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:21:24.766496 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:21:24.766496 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:21:24.766496 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:21:24.766496 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:21:24.775130 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:21:24.775130 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:21:24.775130 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:21:24.775130 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:21:24.775130 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 624/1215] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:21:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:21:24.775639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26c5271-631f-42d3-a6eb-47bf9c4b3276 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:24.861487 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:24.918864 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26c5271-631f-42d3-a6eb-47bf9c4b3276 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:24.966706 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6068a58-fda2-44ab-988c-b130040258a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:24.967598 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6068a58-fda2-44ab-988c-b130040258a7 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:24.967952 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6068a58-fda2-44ab-988c-b130040258a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:24.968278 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6068a58-fda2-44ab-988c-b130040258a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:24.969062 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6068a58-fda2-44ab-988c-b130040258a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:21:24.969387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6068a58-fda2-44ab-988c-b130040258a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:24.971073 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6068a58-fda2-44ab-988c-b130040258a7 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:24.987170 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 592/1216] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:21:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:21:24.993080 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f85855-2035-46de-9b9a-73fdf1101061 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:25.083665 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:25.120194 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f85855-2035-46de-9b9a-73fdf1101061 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:25.167969 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79199803-5e25-409d-b3d0-d7c50774d295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:25.169054 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79199803-5e25-409d-b3d0-d7c50774d295 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:25.169414 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79199803-5e25-409d-b3d0-d7c50774d295 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:25.169783 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79199803-5e25-409d-b3d0-d7c50774d295 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:25.179052 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79199803-5e25-409d-b3d0-d7c50774d295 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:25.180557 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79199803-5e25-409d-b3d0-d7c50774d295 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:25.196099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-79199803-5e25-409d-b3d0-d7c50774d295 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:21:25.196099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:21:25.196099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:21:25.196099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:21:25.196099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:21:25.196099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:21:25.196099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:21:25.196099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:21:25.196099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:21:25.196099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:21:25.196099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:21:25.196099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:21:25.196099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:21:25.196099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:21:25.196099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:21:25.197221 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:21:25.197221 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:21:25.197221 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:21:25.197221 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:21:25.197221 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:21:25.197221 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:21:25.197221 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:21:25.197221 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:21:25.197221 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:21:25.197221 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:21:25.197221 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:21:25.197221 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:21:25.197221 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:21:25.197221 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:21:25.197221 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:21:25.197221 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:21:25.197221 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:21:25.198177 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:21:25.198177 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:21:25.198177 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:21:25.198177 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:21:25.198177 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:21:25.198177 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:21:25.198177 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:21:25.198177 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:21:25.202655 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 625/1217] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:21:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:21:25.203779 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6068a58-fda2-44ab-988c-b130040258a7 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:25.338354 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6068a58-fda2-44ab-988c-b130040258a7 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:25.370062 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:25.385046 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3391bd35-f125-4d3b-b13f-69215f42d6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:25.385748 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391bd35-f125-4d3b-b13f-69215f42d6cc None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:25.386033 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391bd35-f125-4d3b-b13f-69215f42d6cc None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:25.386340 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391bd35-f125-4d3b-b13f-69215f42d6cc None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:25.387157 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3391bd35-f125-4d3b-b13f-69215f42d6cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:21:25.387470 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3391bd35-f125-4d3b-b13f-69215f42d6cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:25.389931 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3391bd35-f125-4d3b-b13f-69215f42d6cc None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:25.410645 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 593/1218] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 21:21:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:21:25.412743 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79199803-5e25-409d-b3d0-d7c50774d295 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:25.537433 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79199803-5e25-409d-b3d0-d7c50774d295 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:25.574897 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:25.582835 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792596ce-b5d4-4ed2-8b5d-372576640b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:25.583625 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792596ce-b5d4-4ed2-8b5d-372576640b10 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:25.583930 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792596ce-b5d4-4ed2-8b5d-372576640b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:25.584214 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792596ce-b5d4-4ed2-8b5d-372576640b10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:25.585358 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792596ce-b5d4-4ed2-8b5d-372576640b10 None admin] RBAC: Adding query filter params (role.id=9dc26e65165b49d6916b54458f847755, scope.project.id=c4940befa9c4405699c72deedc8a2127, user.id=f6143bddc05a46ea972709576c7ad95f) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:21:25.585816 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792596ce-b5d4-4ed2-8b5d-372576640b10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:25.587656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792596ce-b5d4-4ed2-8b5d-372576640b10 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:25.598831 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 626/1219] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:21:25 2022] GET /identity/v3/role_assignments?user.id=f6143bddc05a46ea972709576c7ad95f&scope.project.id=c4940befa9c4405699c72deedc8a2127&role.id=9dc26e65165b49d6916b54458f847755 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:21:28.213050 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3853172-e90f-4b65-8f05-76000fc2e061 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:28.213679 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3853172-e90f-4b65-8f05-76000fc2e061 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:28.213679 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3853172-e90f-4b65-8f05-76000fc2e061 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:28.215275 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 594/1220] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:21:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:21:28.223702 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b04e7-8ce9-4f5c-b806-59ac3ba0a144 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:28.224365 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b04e7-8ce9-4f5c-b806-59ac3ba0a144 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:28.224365 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b04e7-8ce9-4f5c-b806-59ac3ba0a144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:28.249390 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-2c9b04e7-8ce9-4f5c-b806-59ac3ba0a144 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:21:28.250678 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2c9b04e7-8ce9-4f5c-b806-59ac3ba0a144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:21:28.351644 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 627/1221] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:21:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:21:28.387320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3853172-e90f-4b65-8f05-76000fc2e061 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:28.529217 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3853172-e90f-4b65-8f05-76000fc2e061 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:28.576885 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9265d2b5-bced-496c-a209-6567f4d43714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:28.577996 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265d2b5-bced-496c-a209-6567f4d43714 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:28.578463 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265d2b5-bced-496c-a209-6567f4d43714 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:28.578858 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265d2b5-bced-496c-a209-6567f4d43714 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:28.596547 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9265d2b5-bced-496c-a209-6567f4d43714 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:21:28.611083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-9265d2b5-bced-496c-a209-6567f4d43714 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:21:28.611083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:21:28.611083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:21:28.611083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:21:28.611083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:21:28.611083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:21:28.611083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:21:28.611083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:21:28.611083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:21:28.611083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:21:28.611083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:21:28.611083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:21:28.611083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:21:28.611083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:21:28.611083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:21:28.611083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:21:28.613220 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:21:28.613220 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:21:28.613220 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:21:28.613220 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:21:28.613220 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:21:28.613220 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:21:28.613220 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:21:28.613220 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:21:28.613220 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:21:28.613220 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:21:28.613220 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:21:28.613220 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:21:28.613220 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:21:28.613220 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:21:28.613220 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:21:28.613220 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:21:28.616431 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:21:28.616431 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:21:28.616431 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:21:28.616431 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:21:28.616431 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:21:28.616431 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:21:28.616431 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:21:28.616431 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:21:28.616431 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:21:28.616431 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:21:28.616431 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:21:28.616431 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:21:28.616431 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 595/1222] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 21:21:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:21:28.623680 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b04e7-8ce9-4f5c-b806-59ac3ba0a144 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:28.760110 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b04e7-8ce9-4f5c-b806-59ac3ba0a144 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:28.790160 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:28.802892 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54327686-db96-4845-b3a3-1a6b6603af56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:28.803585 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54327686-db96-4845-b3a3-1a6b6603af56 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:28.803987 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54327686-db96-4845-b3a3-1a6b6603af56 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:28.804144 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54327686-db96-4845-b3a3-1a6b6603af56 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:28.804908 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54327686-db96-4845-b3a3-1a6b6603af56 None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:21:28.805219 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54327686-db96-4845-b3a3-1a6b6603af56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:28.806747 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54327686-db96-4845-b3a3-1a6b6603af56 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:28.816416 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 628/1223] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 21:21:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:21:28.822575 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9265d2b5-bced-496c-a209-6567f4d43714 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:28.874696 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:28.946942 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9265d2b5-bced-496c-a209-6567f4d43714 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:28.987452 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f65e4d2-46b7-41f5-a581-511b21d0f32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:28.988374 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f65e4d2-46b7-41f5-a581-511b21d0f32f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:28.988690 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f65e4d2-46b7-41f5-a581-511b21d0f32f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:28.989019 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f65e4d2-46b7-41f5-a581-511b21d0f32f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:29.003415 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f65e4d2-46b7-41f5-a581-511b21d0f32f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:29.005154 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f65e4d2-46b7-41f5-a581-511b21d0f32f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:29.030393 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-6f65e4d2-46b7-41f5-a581-511b21d0f32f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:21:29.030393 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:21:29.030393 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:21:29.030393 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:21:29.030393 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:21:29.030393 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:21:29.030393 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:21:29.030393 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:21:29.030393 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:21:29.030393 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:21:29.030393 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:21:29.030393 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:21:29.030393 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:21:29.030393 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:21:29.030393 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:21:29.032031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:21:29.032031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:21:29.032031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:21:29.032031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:21:29.032031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:21:29.032031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:21:29.032031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:21:29.032031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:21:29.032031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:21:29.032031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:21:29.032031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:21:29.032031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:21:29.032031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:21:29.032031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:21:29.032031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:21:29.032031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:21:29.032031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:21:29.034031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:21:29.034031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:21:29.034031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:21:29.034031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:21:29.034031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 596/1224] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:21:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:21:29.039126 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54327686-db96-4845-b3a3-1a6b6603af56 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:29.179416 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54327686-db96-4845-b3a3-1a6b6603af56 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:29.230016 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b240fc35-2b05-436d-a48b-8345ee269aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:29.231279 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b240fc35-2b05-436d-a48b-8345ee269aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:29.231637 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b240fc35-2b05-436d-a48b-8345ee269aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:29.232183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b240fc35-2b05-436d-a48b-8345ee269aa4 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:29.233758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b240fc35-2b05-436d-a48b-8345ee269aa4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:21:29.234155 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b240fc35-2b05-436d-a48b-8345ee269aa4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:29.235938 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b240fc35-2b05-436d-a48b-8345ee269aa4 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:29.259623 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 629/1225] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:21:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:21:29.268420 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f65e4d2-46b7-41f5-a581-511b21d0f32f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:29.412739 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:29.428311 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f65e4d2-46b7-41f5-a581-511b21d0f32f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:29.485804 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc449f25-45ff-4beb-8336-6e74ca0196c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:29.486601 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc449f25-45ff-4beb-8336-6e74ca0196c9 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:29.486892 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc449f25-45ff-4beb-8336-6e74ca0196c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:29.487159 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc449f25-45ff-4beb-8336-6e74ca0196c9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:29.496502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc449f25-45ff-4beb-8336-6e74ca0196c9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:29.498579 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc449f25-45ff-4beb-8336-6e74ca0196c9 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:29.516989 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-cc449f25-45ff-4beb-8336-6e74ca0196c9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 21:21:29.516989 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:21:29.516989 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:21:29.516989 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:21:29.516989 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:21:29.516989 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:21:29.516989 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:21:29.516989 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:21:29.516989 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:21:29.516989 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:21:29.516989 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:21:29.516989 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:21:29.516989 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:21:29.516989 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:21:29.516989 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:21:29.516989 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:21:29.519778 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:21:29.519778 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:21:29.519778 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:21:29.519778 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:21:29.519778 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:21:29.519778 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:21:29.519778 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:21:29.519778 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:21:29.519778 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:21:29.519778 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:21:29.519778 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:21:29.519778 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:21:29.519778 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:21:29.519778 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:21:29.519778 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:21:29.519778 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:21:29.519778 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:21:29.519778 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:21:29.521710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:21:29.521710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:21:29.521710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:21:29.521710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:21:29.521710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:21:29.521710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:21:29.521710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:21:29.521710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:21:29.521710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 21:21:29.521710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:21:29.521710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 597/1226] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:21:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:21:29.524293 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b240fc35-2b05-436d-a48b-8345ee269aa4 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:29.590224 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:29.673618 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b240fc35-2b05-436d-a48b-8345ee269aa4 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:29.730952 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47a23a-b5c7-498b-a093-5aa2e992305d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:29.732062 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47a23a-b5c7-498b-a093-5aa2e992305d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:29.733363 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47a23a-b5c7-498b-a093-5aa2e992305d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:29.733535 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47a23a-b5c7-498b-a093-5aa2e992305d None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:29.734731 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f47a23a-b5c7-498b-a093-5aa2e992305d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:21:29.735027 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f47a23a-b5c7-498b-a093-5aa2e992305d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:29.737146 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f47a23a-b5c7-498b-a093-5aa2e992305d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:29.756959 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 630/1227] 10.222.0.65 () {56 vars in 1143 bytes} [Thu May 19 21:21:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:21:29.763154 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc449f25-45ff-4beb-8336-6e74ca0196c9 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:29.885887 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc449f25-45ff-4beb-8336-6e74ca0196c9 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:29.937542 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce5c9c-d014-44e8-822c-e29b7806a0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:29.938558 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce5c9c-d014-44e8-822c-e29b7806a0bc None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:29.938888 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce5c9c-d014-44e8-822c-e29b7806a0bc None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:29.939197 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce5c9c-d014-44e8-822c-e29b7806a0bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:29.954541 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcce5c9c-d014-44e8-822c-e29b7806a0bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:29.956338 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcce5c9c-d014-44e8-822c-e29b7806a0bc None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:29.977141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-bcce5c9c-d014-44e8-822c-e29b7806a0bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:21:29.977141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:21:29.977141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:21:29.977141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:21:29.977141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:21:29.977141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:21:29.977141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:21:29.977141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:21:29.977141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:21:29.977141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:21:29.977141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:21:29.977141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:21:29.977141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:21:29.977141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:21:29.977141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:21:29.978202 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:21:29.978202 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:21:29.978202 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:21:29.978202 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:21:29.978202 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:21:29.978202 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:21:29.978202 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:21:29.978202 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:21:29.978202 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:21:29.978202 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:21:29.978202 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:21:29.978202 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:21:29.978202 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:21:29.978202 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:21:29.978202 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:21:29.978202 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:21:29.978202 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:21:29.979143 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:21:29.979143 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:21:29.979143 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:21:29.979143 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:21:29.979143 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 598/1228] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:21:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:21:29.985208 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47a23a-b5c7-498b-a093-5aa2e992305d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:30.142464 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47a23a-b5c7-498b-a093-5aa2e992305d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:30.146629 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:30.192898 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5984edc-7621-4b9e-9909-1e80e148e85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:30.193873 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5984edc-7621-4b9e-9909-1e80e148e85a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:30.194413 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5984edc-7621-4b9e-9909-1e80e148e85a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:30.194780 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5984edc-7621-4b9e-9909-1e80e148e85a None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:30.195942 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5984edc-7621-4b9e-9909-1e80e148e85a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:21:30.196455 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5984edc-7621-4b9e-9909-1e80e148e85a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:30.198323 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5984edc-7621-4b9e-9909-1e80e148e85a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:30.215750 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 631/1229] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:21:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:21:30.222653 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce5c9c-d014-44e8-822c-e29b7806a0bc None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:30.235425 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:30.386347 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce5c9c-d014-44e8-822c-e29b7806a0bc None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:30.432584 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ce7a6-051c-494b-8434-4346facfa44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:30.433504 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ce7a6-051c-494b-8434-4346facfa44f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:30.433885 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ce7a6-051c-494b-8434-4346facfa44f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:30.434346 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ce7a6-051c-494b-8434-4346facfa44f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:30.442948 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ce7a6-051c-494b-8434-4346facfa44f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:30.444440 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ce7a6-051c-494b-8434-4346facfa44f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:30.460420 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-766ce7a6-051c-494b-8434-4346facfa44f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:21:30.460420 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:21:30.460420 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:21:30.460420 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:21:30.460420 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:21:30.460420 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:21:30.460420 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:21:30.460420 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:21:30.460420 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:21:30.460420 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:21:30.460420 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:21:30.460420 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:21:30.460420 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:21:30.460420 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:21:30.460420 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:21:30.462327 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:21:30.462327 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:21:30.462327 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:21:30.462327 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:21:30.462327 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:21:30.462327 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:21:30.462327 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:21:30.462327 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:21:30.462327 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:21:30.462327 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:21:30.462327 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:21:30.462327 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:21:30.462327 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:21:30.462327 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:21:30.462327 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:21:30.462327 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:21:30.462327 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:21:30.463940 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:21:30.463940 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:21:30.463940 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:21:30.463940 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:21:30.463940 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:21:30.463940 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:21:30.463940 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:21:30.463940 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:21:30.463940 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 599/1230] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:21:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:21:30.470329 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5984edc-7621-4b9e-9909-1e80e148e85a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:30.591243 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5984edc-7621-4b9e-9909-1e80e148e85a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:30.638876 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438ec3af-ed5f-4c59-85fc-2feae6fc1825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:30.639694 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ec3af-ed5f-4c59-85fc-2feae6fc1825 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:30.640101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ec3af-ed5f-4c59-85fc-2feae6fc1825 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:30.640533 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ec3af-ed5f-4c59-85fc-2feae6fc1825 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:30.642130 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438ec3af-ed5f-4c59-85fc-2feae6fc1825 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:21:30.642800 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438ec3af-ed5f-4c59-85fc-2feae6fc1825 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:30.645272 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438ec3af-ed5f-4c59-85fc-2feae6fc1825 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:30.665704 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 632/1231] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 21:21:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:21:30.674728 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ce7a6-051c-494b-8434-4346facfa44f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:30.757023 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:30.794026 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ce7a6-051c-494b-8434-4346facfa44f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:30.842477 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd784da-a65d-49a9-b15a-2ceb16719a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:30.843467 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd784da-a65d-49a9-b15a-2ceb16719a76 None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:30.843756 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd784da-a65d-49a9-b15a-2ceb16719a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:30.844065 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd784da-a65d-49a9-b15a-2ceb16719a76 None admin] PATH_INFO: `/v3/projects/c4940befa9c4405699c72deedc8a2127/users/f6143bddc05a46ea972709576c7ad95f/roles/9dc26e65165b49d6916b54458f847755` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:30.866664 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd784da-a65d-49a9-b15a-2ceb16719a76 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4940befa9c4405699c72deedc8a2127, user_id=f6143bddc05a46ea972709576c7ad95f, role_id=9dc26e65165b49d6916b54458f847755)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:30.868197 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd784da-a65d-49a9-b15a-2ceb16719a76 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:30.885385 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 600/1232] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 21:21:30 2022] PUT /identity/v3/projects/c4940befa9c4405699c72deedc8a2127/users/f6143bddc05a46ea972709576c7ad95f/roles/9dc26e65165b49d6916b54458f847755 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:21:33.169035 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ab509-d7a7-4f90-8dea-2c37b82de8d9 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:33.169856 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ab509-d7a7-4f90-8dea-2c37b82de8d9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:33.169856 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ab509-d7a7-4f90-8dea-2c37b82de8d9 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:33.171576 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 633/1233] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:21:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:21:33.177685 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42bda5c-b743-47df-a338-f5438619af3c None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:33.178355 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42bda5c-b743-47df-a338-f5438619af3c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:33.178355 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42bda5c-b743-47df-a338-f5438619af3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:33.197115 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-c42bda5c-b743-47df-a338-f5438619af3c None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:21:33.198802 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c42bda5c-b743-47df-a338-f5438619af3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:21:33.279094 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 601/1234] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:21:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:21:33.304338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ab509-d7a7-4f90-8dea-2c37b82de8d9 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:33.333520 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:33.441548 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ab509-d7a7-4f90-8dea-2c37b82de8d9 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:33.499162 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b774d2-1237-43c4-af4b-79b0a2ef380e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:33.499984 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b774d2-1237-43c4-af4b-79b0a2ef380e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:33.500274 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b774d2-1237-43c4-af4b-79b0a2ef380e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:33.500547 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b774d2-1237-43c4-af4b-79b0a2ef380e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:33.515008 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b774d2-1237-43c4-af4b-79b0a2ef380e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:21:33.526859 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-b7b774d2-1237-43c4-af4b-79b0a2ef380e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 21:21:33.526859 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:21:33.526859 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:21:33.526859 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:21:33.526859 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:21:33.526859 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:21:33.526859 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:21:33.526859 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:21:33.526859 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:21:33.526859 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:21:33.526859 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:21:33.526859 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:21:33.526859 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:21:33.526859 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:21:33.526859 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:21:33.526859 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:21:33.529122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:21:33.529122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:21:33.529122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:21:33.529122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:21:33.529122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:21:33.529122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:21:33.529122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:21:33.529122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:21:33.529122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:21:33.529122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:21:33.529122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:21:33.529122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:21:33.529122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:21:33.529122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:21:33.529122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:21:33.529122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:21:33.530875 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:21:33.530875 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:21:33.530875 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:21:33.530875 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:21:33.530875 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:21:33.530875 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:21:33.530875 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:21:33.530875 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:21:33.530875 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:21:33.530875 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:21:33.530875 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 21:21:33.530875 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:21:33.530875 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 634/1235] 10.222.0.65 () {56 vars in 1102 bytes} [Thu May 19 21:21:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:21:33.542144 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42bda5c-b743-47df-a338-f5438619af3c None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:33.615048 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:33.698951 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42bda5c-b743-47df-a338-f5438619af3c None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:33.749559 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97750ee5-a507-4ec2-8969-85219d2ff29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:33.750269 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97750ee5-a507-4ec2-8969-85219d2ff29d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:33.750576 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97750ee5-a507-4ec2-8969-85219d2ff29d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:33.750850 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97750ee5-a507-4ec2-8969-85219d2ff29d None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:33.751614 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97750ee5-a507-4ec2-8969-85219d2ff29d None admin] RBAC: Adding query filter params (name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:21:33.751923 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97750ee5-a507-4ec2-8969-85219d2ff29d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:33.753330 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97750ee5-a507-4ec2-8969-85219d2ff29d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:33.765083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 602/1236] 10.222.0.65 () {56 vars in 1103 bytes} [Thu May 19 21:21:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:21:33.772128 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b774d2-1237-43c4-af4b-79b0a2ef380e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:33.898230 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:33.930311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b774d2-1237-43c4-af4b-79b0a2ef380e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:33.991044 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8493f2-a6cf-4020-87ee-d848ef306a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:33.992209 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8493f2-a6cf-4020-87ee-d848ef306a08 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:33.992666 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8493f2-a6cf-4020-87ee-d848ef306a08 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:33.993101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8493f2-a6cf-4020-87ee-d848ef306a08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:34.010436 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8493f2-a6cf-4020-87ee-d848ef306a08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:34.012599 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8493f2-a6cf-4020-87ee-d848ef306a08 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:34.034580 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-0a8493f2-a6cf-4020-87ee-d848ef306a08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:21:34.034580 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:21:34.034580 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:21:34.034580 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:21:34.034580 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:21:34.034580 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:21:34.034580 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:21:34.034580 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:21:34.034580 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:21:34.034580 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:21:34.034580 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:21:34.034580 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:21:34.034580 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:21:34.034580 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:21:34.034580 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:21:34.036832 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:21:34.036832 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:21:34.036832 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:21:34.036832 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:21:34.036832 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:21:34.036832 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:21:34.036832 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:21:34.036832 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:21:34.036832 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:21:34.036832 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:21:34.036832 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:21:34.036832 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:21:34.036832 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:21:34.036832 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:21:34.036832 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:21:34.036832 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:21:34.036832 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:21:34.038813 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:21:34.038813 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:21:34.038813 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:21:34.038813 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:21:34.038813 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 635/1237] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:21:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:21:34.043430 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97750ee5-a507-4ec2-8969-85219d2ff29d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:34.192212 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97750ee5-a507-4ec2-8969-85219d2ff29d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:34.206126 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:34.248837 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb291cb-f4f8-4c3f-af24-8a00a324aa66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:34.249425 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb291cb-f4f8-4c3f-af24-8a00a324aa66 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:34.249720 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb291cb-f4f8-4c3f-af24-8a00a324aa66 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:34.249970 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb291cb-f4f8-4c3f-af24-8a00a324aa66 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:34.250739 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb291cb-f4f8-4c3f-af24-8a00a324aa66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:21:34.251037 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb291cb-f4f8-4c3f-af24-8a00a324aa66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:34.252441 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb291cb-f4f8-4c3f-af24-8a00a324aa66 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:34.265426 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 603/1238] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:21:34.272131 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8493f2-a6cf-4020-87ee-d848ef306a08 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:34.401378 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8493f2-a6cf-4020-87ee-d848ef306a08 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:34.453206 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de9481-a30c-4b48-83be-76cd33690725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:34.454164 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de9481-a30c-4b48-83be-76cd33690725 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:34.454581 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de9481-a30c-4b48-83be-76cd33690725 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:34.454977 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de9481-a30c-4b48-83be-76cd33690725 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:34.459386 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:34.466737 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de9481-a30c-4b48-83be-76cd33690725 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:34.469226 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de9481-a30c-4b48-83be-76cd33690725 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:34.494879 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-00de9481-a30c-4b48-83be-76cd33690725 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 21:21:34.494879 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:21:34.494879 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:21:34.494879 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:21:34.494879 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:21:34.494879 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:21:34.494879 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:21:34.494879 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:21:34.494879 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:21:34.494879 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:21:34.494879 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:21:34.494879 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:21:34.494879 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:21:34.494879 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:21:34.494879 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:21:34.494879 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:21:34.496758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:21:34.496758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:21:34.496758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:21:34.496758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:21:34.496758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:21:34.496758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:21:34.496758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:21:34.496758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:21:34.496758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:21:34.496758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:21:34.496758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:21:34.496758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:21:34.496758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:21:34.496758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:21:34.496758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:21:34.496758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:21:34.496758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:21:34.496758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:21:34.498714 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:21:34.498714 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:21:34.498714 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:21:34.498714 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:21:34.498714 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:21:34.498714 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:21:34.498714 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:21:34.498714 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:21:34.498714 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 21:21:34.498714 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:21:34.498714 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 636/1239] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:21:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:21:34.505086 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb291cb-f4f8-4c3f-af24-8a00a324aa66 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:34.660382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb291cb-f4f8-4c3f-af24-8a00a324aa66 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:34.706278 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c7459e-b07d-4416-8e31-08c39457e2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:34.707044 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7459e-b07d-4416-8e31-08c39457e2d2 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:34.707377 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7459e-b07d-4416-8e31-08c39457e2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:34.707683 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7459e-b07d-4416-8e31-08c39457e2d2 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:34.712598 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c7459e-b07d-4416-8e31-08c39457e2d2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:21:34.712598 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c7459e-b07d-4416-8e31-08c39457e2d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:34.712598 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c7459e-b07d-4416-8e31-08c39457e2d2 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:34.735215 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 604/1240] 10.222.0.65 () {56 vars in 1143 bytes} [Thu May 19 21:21:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:21:34.741337 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de9481-a30c-4b48-83be-76cd33690725 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:34.896988 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de9481-a30c-4b48-83be-76cd33690725 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:34.940106 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7f605-586e-4239-bf77-f24749746888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:34.940998 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7f605-586e-4239-bf77-f24749746888 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:34.941287 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7f605-586e-4239-bf77-f24749746888 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:34.941603 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7f605-586e-4239-bf77-f24749746888 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:35.023912 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:35.032038 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de7f605-586e-4239-bf77-f24749746888 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:35.033638 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de7f605-586e-4239-bf77-f24749746888 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:35.171695 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 637/1241] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:21:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 21:21:35.177570 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c7459e-b07d-4416-8e31-08c39457e2d2 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:35.213061 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:35.317799 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c7459e-b07d-4416-8e31-08c39457e2d2 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:35.368960 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9e65e-3009-4592-a88e-adce08b25f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:35.369791 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9e65e-3009-4592-a88e-adce08b25f82 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:35.370077 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9e65e-3009-4592-a88e-adce08b25f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:35.370386 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9e65e-3009-4592-a88e-adce08b25f82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:35.383154 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9e65e-3009-4592-a88e-adce08b25f82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:35.384923 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9e65e-3009-4592-a88e-adce08b25f82 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:35.409613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-5ac9e65e-3009-4592-a88e-adce08b25f82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:21:35.409613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:21:35.409613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:21:35.409613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:21:35.409613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:21:35.409613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:21:35.409613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:21:35.409613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:21:35.409613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:21:35.409613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:21:35.409613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:21:35.409613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:21:35.409613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:21:35.409613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:21:35.409613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:21:35.410944 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:21:35.410944 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:21:35.410944 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:21:35.410944 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:21:35.410944 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:21:35.410944 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:21:35.410944 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:21:35.410944 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:21:35.410944 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:21:35.410944 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:21:35.410944 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:21:35.410944 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:21:35.410944 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:21:35.410944 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:21:35.410944 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:21:35.410944 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:21:35.410944 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:21:35.411971 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:21:35.411971 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:21:35.411971 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:21:35.411971 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:21:35.411971 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 605/1242] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:21:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:21:35.417586 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7f605-586e-4239-bf77-f24749746888 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:35.556826 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:35.565579 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7f605-586e-4239-bf77-f24749746888 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:35.615180 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b101fe2-a949-4d37-b8b4-164d40b6bdde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:35.615903 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b101fe2-a949-4d37-b8b4-164d40b6bdde None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:35.616228 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b101fe2-a949-4d37-b8b4-164d40b6bdde None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:35.616589 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b101fe2-a949-4d37-b8b4-164d40b6bdde None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:35.617353 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b101fe2-a949-4d37-b8b4-164d40b6bdde None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:21:35.617665 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b101fe2-a949-4d37-b8b4-164d40b6bdde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:35.619234 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b101fe2-a949-4d37-b8b4-164d40b6bdde None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:35.642524 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 638/1243] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:21:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:21:35.646001 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9e65e-3009-4592-a88e-adce08b25f82 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:35.768464 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:35.803014 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9e65e-3009-4592-a88e-adce08b25f82 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:35.855823 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4bfdbb-2c41-4232-8aa4-ad6b34859e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:35.856784 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4bfdbb-2c41-4232-8aa4-ad6b34859e89 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:35.857421 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4bfdbb-2c41-4232-8aa4-ad6b34859e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:35.857724 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4bfdbb-2c41-4232-8aa4-ad6b34859e89 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:35.866153 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4bfdbb-2c41-4232-8aa4-ad6b34859e89 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:35.867731 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4bfdbb-2c41-4232-8aa4-ad6b34859e89 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:35.893221 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-ca4bfdbb-2c41-4232-8aa4-ad6b34859e89 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:21:35.893221 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:21:35.893221 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:21:35.893221 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:21:35.893221 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:21:35.893221 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:21:35.893221 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:21:35.893221 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:21:35.893221 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:21:35.893221 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:21:35.893221 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:21:35.893221 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:21:35.893221 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:21:35.893221 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:21:35.893221 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:21:35.894867 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:21:35.894867 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:21:35.894867 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:21:35.894867 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:21:35.894867 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:21:35.894867 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:21:35.894867 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:21:35.894867 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:21:35.894867 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:21:35.894867 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:21:35.894867 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:21:35.894867 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:21:35.894867 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:21:35.894867 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:21:35.894867 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:21:35.894867 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:21:35.894867 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:21:35.896284 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:21:35.896284 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:21:35.896284 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:21:35.896284 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:21:35.896284 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:21:35.896284 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:21:35.896284 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:21:35.896284 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:21:35.902419 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 606/1244] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:21:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:21:35.902697 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b101fe2-a949-4d37-b8b4-164d40b6bdde None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:36.046338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b101fe2-a949-4d37-b8b4-164d40b6bdde None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:36.095788 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0907ea0-8dda-4607-8ba8-bcb80cd5c07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:36.096545 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0907ea0-8dda-4607-8ba8-bcb80cd5c07b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:36.096828 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0907ea0-8dda-4607-8ba8-bcb80cd5c07b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:36.097150 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0907ea0-8dda-4607-8ba8-bcb80cd5c07b None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:36.098169 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0907ea0-8dda-4607-8ba8-bcb80cd5c07b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:21:36.098534 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0907ea0-8dda-4607-8ba8-bcb80cd5c07b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:36.100066 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0907ea0-8dda-4607-8ba8-bcb80cd5c07b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:36.115719 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 639/1245] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 21:21:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:21:36.122537 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4bfdbb-2c41-4232-8aa4-ad6b34859e89 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:36.270444 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4bfdbb-2c41-4232-8aa4-ad6b34859e89 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:36.310308 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f5cfb-1c38-4183-a606-f46afb80309f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:36.310308 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f5cfb-1c38-4183-a606-f46afb80309f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:36.310308 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f5cfb-1c38-4183-a606-f46afb80309f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:36.310308 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f5cfb-1c38-4183-a606-f46afb80309f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:36.311924 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8f5cfb-1c38-4183-a606-f46afb80309f None admin] RBAC: Adding query filter params (role.id=9dc26e65165b49d6916b54458f847755, scope.project.id=c4940befa9c4405699c72deedc8a2127, user.id=f6143bddc05a46ea972709576c7ad95f) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:21:36.312277 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8f5cfb-1c38-4183-a606-f46afb80309f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:36.313867 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8f5cfb-1c38-4183-a606-f46afb80309f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:36.321717 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 607/1246] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:21:36 2022] GET /identity/v3/role_assignments?user.id=f6143bddc05a46ea972709576c7ad95f&scope.project.id=c4940befa9c4405699c72deedc8a2127&role.id=9dc26e65165b49d6916b54458f847755 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:21:38.841125 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3315568-e272-4add-abed-f93a471d4f0c None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:38.841947 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3315568-e272-4add-abed-f93a471d4f0c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:38.841947 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3315568-e272-4add-abed-f93a471d4f0c None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:38.845420 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 640/1247] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:21:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:21:38.849711 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed109dc-3671-400f-b7b5-42106fb24a2f None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:38.850301 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed109dc-3671-400f-b7b5-42106fb24a2f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:38.850301 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed109dc-3671-400f-b7b5-42106fb24a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:38.859732 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:38.874347 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-9ed109dc-3671-400f-b7b5-42106fb24a2f None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:21:38.875878 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9ed109dc-3671-400f-b7b5-42106fb24a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:21:38.967723 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 608/1248] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:21:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 21:21:38.991849 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3315568-e272-4add-abed-f93a471d4f0c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:39.013097 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:39.117659 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3315568-e272-4add-abed-f93a471d4f0c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:39.157575 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf4122-75b1-44a5-89a5-49de11b99148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:39.158458 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf4122-75b1-44a5-89a5-49de11b99148 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:39.158746 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf4122-75b1-44a5-89a5-49de11b99148 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:39.159025 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf4122-75b1-44a5-89a5-49de11b99148 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:39.172183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cf4122-75b1-44a5-89a5-49de11b99148 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:21:39.179933 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-24cf4122-75b1-44a5-89a5-49de11b99148 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:21:39.179933 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:21:39.179933 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:21:39.179933 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:21:39.179933 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:21:39.179933 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:21:39.179933 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:21:39.179933 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:21:39.179933 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:21:39.179933 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:21:39.179933 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:21:39.179933 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:21:39.179933 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:21:39.179933 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:21:39.179933 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:21:39.179933 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:21:39.181022 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:21:39.181022 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:21:39.181022 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:21:39.181022 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:21:39.181022 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:21:39.181022 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:21:39.181022 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:21:39.181022 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:21:39.181022 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:21:39.181022 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:21:39.181022 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:21:39.181022 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:21:39.181022 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:21:39.181022 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:21:39.181022 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:21:39.181022 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:21:39.182343 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:21:39.182343 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:21:39.182343 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:21:39.182343 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:21:39.182343 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:21:39.182343 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:21:39.182343 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:21:39.182343 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:21:39.182343 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:21:39.182343 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:21:39.182343 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:21:39.182343 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:21:39.182343 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 641/1249] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 21:21:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:21:39.187779 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed109dc-3671-400f-b7b5-42106fb24a2f None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:39.300557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed109dc-3671-400f-b7b5-42106fb24a2f None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:39.338391 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:39.344128 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251eb538-2a3e-4971-bb21-d45ed995443c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:39.344870 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251eb538-2a3e-4971-bb21-d45ed995443c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:39.345399 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251eb538-2a3e-4971-bb21-d45ed995443c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:39.345755 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251eb538-2a3e-4971-bb21-d45ed995443c None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:39.346738 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251eb538-2a3e-4971-bb21-d45ed995443c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:21:39.347185 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251eb538-2a3e-4971-bb21-d45ed995443c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:39.348580 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251eb538-2a3e-4971-bb21-d45ed995443c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:39.358876 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 609/1250] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:21:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:21:39.365362 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf4122-75b1-44a5-89a5-49de11b99148 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:39.452663 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:39.476387 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf4122-75b1-44a5-89a5-49de11b99148 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:39.514141 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f780a-aadd-4d6a-845b-1538446e8517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:39.515108 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f780a-aadd-4d6a-845b-1538446e8517 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:39.515415 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f780a-aadd-4d6a-845b-1538446e8517 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:39.515755 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f780a-aadd-4d6a-845b-1538446e8517 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:39.529744 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037f780a-aadd-4d6a-845b-1538446e8517 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:39.531406 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037f780a-aadd-4d6a-845b-1538446e8517 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:39.554195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-037f780a-aadd-4d6a-845b-1538446e8517 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:21:39.554195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:21:39.554195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:21:39.554195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:21:39.554195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:21:39.554195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:21:39.554195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:21:39.554195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:21:39.554195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:21:39.554195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:21:39.554195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:21:39.554195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:21:39.554195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:21:39.554195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:21:39.554195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:21:39.555392 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:21:39.555392 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:21:39.555392 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:21:39.555392 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:21:39.555392 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:21:39.555392 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:21:39.555392 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:21:39.555392 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:21:39.555392 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:21:39.555392 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:21:39.555392 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:21:39.555392 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:21:39.555392 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:21:39.555392 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:21:39.555392 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:21:39.555392 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:21:39.555392 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:21:39.556519 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:21:39.556519 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:21:39.556519 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:21:39.556519 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:21:39.562581 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 642/1251] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:21:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:21:39.563943 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251eb538-2a3e-4971-bb21-d45ed995443c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:39.700004 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251eb538-2a3e-4971-bb21-d45ed995443c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:39.742436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37703302-a885-4854-a799-a7cbdef1f057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:39.742927 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37703302-a885-4854-a799-a7cbdef1f057 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:39.743205 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37703302-a885-4854-a799-a7cbdef1f057 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:39.743470 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37703302-a885-4854-a799-a7cbdef1f057 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:39.744262 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37703302-a885-4854-a799-a7cbdef1f057 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:21:39.744567 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37703302-a885-4854-a799-a7cbdef1f057 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:39.746108 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37703302-a885-4854-a799-a7cbdef1f057 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:39.761482 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 610/1252] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:21:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:21:39.767664 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f780a-aadd-4d6a-845b-1538446e8517 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:39.878154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f780a-aadd-4d6a-845b-1538446e8517 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:39.920196 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:39.930930 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2fa542-c6f8-4d1b-a9b9-716b63a3c9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:39.931845 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2fa542-c6f8-4d1b-a9b9-716b63a3c9a5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:39.932189 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2fa542-c6f8-4d1b-a9b9-716b63a3c9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:39.932648 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2fa542-c6f8-4d1b-a9b9-716b63a3c9a5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:39.944611 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2fa542-c6f8-4d1b-a9b9-716b63a3c9a5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:39.946178 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2fa542-c6f8-4d1b-a9b9-716b63a3c9a5 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:39.963656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-dd2fa542-c6f8-4d1b-a9b9-716b63a3c9a5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 21:21:39.963656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:21:39.963656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:21:39.963656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:21:39.963656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:21:39.963656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:21:39.963656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:21:39.963656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:21:39.963656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:21:39.963656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:21:39.963656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:21:39.963656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:21:39.963656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:21:39.963656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:21:39.963656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:21:39.963656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:21:39.965006 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:21:39.965006 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:21:39.965006 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:21:39.965006 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:21:39.965006 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:21:39.965006 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:21:39.965006 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:21:39.965006 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:21:39.965006 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:21:39.965006 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:21:39.965006 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:21:39.965006 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:21:39.965006 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:21:39.965006 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:21:39.965006 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:21:39.965006 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:21:39.965006 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:21:39.965006 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:21:39.966492 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:21:39.966492 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:21:39.966492 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:21:39.966492 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:21:39.966492 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:21:39.966492 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:21:39.966492 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:21:39.966492 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:21:39.966492 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 21:21:39.966492 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:21:39.966492 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 643/1253] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:21:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 21:21:39.971545 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37703302-a885-4854-a799-a7cbdef1f057 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:40.037979 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:40.094037 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37703302-a885-4854-a799-a7cbdef1f057 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:40.138849 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9be29-e6ec-498a-9da6-10d8b7a8f9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:40.139794 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9be29-e6ec-498a-9da6-10d8b7a8f9f4 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:40.140085 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9be29-e6ec-498a-9da6-10d8b7a8f9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:40.140430 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9be29-e6ec-498a-9da6-10d8b7a8f9f4 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:40.141512 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b9be29-e6ec-498a-9da6-10d8b7a8f9f4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:21:40.141974 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b9be29-e6ec-498a-9da6-10d8b7a8f9f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:40.143866 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b9be29-e6ec-498a-9da6-10d8b7a8f9f4 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:40.166696 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 611/1254] 10.222.0.65 () {56 vars in 1143 bytes} [Thu May 19 21:21:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:21:40.172963 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2fa542-c6f8-4d1b-a9b9-716b63a3c9a5 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:40.298886 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2fa542-c6f8-4d1b-a9b9-716b63a3c9a5 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:40.339120 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e9d6be-b0e0-408a-b914-174eb7f22f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:40.339947 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e9d6be-b0e0-408a-b914-174eb7f22f1b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:40.340218 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e9d6be-b0e0-408a-b914-174eb7f22f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:40.340495 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e9d6be-b0e0-408a-b914-174eb7f22f1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:40.401153 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:40.421942 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e9d6be-b0e0-408a-b914-174eb7f22f1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:40.423515 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e9d6be-b0e0-408a-b914-174eb7f22f1b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:40.538773 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 644/1255] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:21:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 21:21:40.545889 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9be29-e6ec-498a-9da6-10d8b7a8f9f4 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:40.682478 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9be29-e6ec-498a-9da6-10d8b7a8f9f4 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:40.698910 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:40.724056 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67dd398-3639-4f05-be38-045a11419833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:40.724891 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67dd398-3639-4f05-be38-045a11419833 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:40.725176 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67dd398-3639-4f05-be38-045a11419833 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:40.725453 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67dd398-3639-4f05-be38-045a11419833 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:40.738099 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67dd398-3639-4f05-be38-045a11419833 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:40.739626 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67dd398-3639-4f05-be38-045a11419833 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:40.759258 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-f67dd398-3639-4f05-be38-045a11419833 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:21:40.759258 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:21:40.759258 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:21:40.759258 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:21:40.759258 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:21:40.759258 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:21:40.759258 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:21:40.759258 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:21:40.759258 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:21:40.759258 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:21:40.759258 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:21:40.759258 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:21:40.759258 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:21:40.759258 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:21:40.759258 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:21:40.760320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:21:40.760320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:21:40.760320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:21:40.760320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:21:40.760320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:21:40.760320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:21:40.760320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:21:40.760320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:21:40.760320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:21:40.760320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:21:40.760320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:21:40.760320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:21:40.760320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:21:40.760320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:21:40.760320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:21:40.760320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:21:40.760320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:21:40.761249 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:21:40.761249 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:21:40.761249 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:21:40.761249 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:21:40.761249 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 612/1256] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:21:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:21:40.767170 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e9d6be-b0e0-408a-b914-174eb7f22f1b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:40.870692 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:40.884923 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e9d6be-b0e0-408a-b914-174eb7f22f1b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:40.922213 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a651d-2939-4383-b7df-007a3ca1cda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:40.922928 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a651d-2939-4383-b7df-007a3ca1cda0 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:40.923201 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a651d-2939-4383-b7df-007a3ca1cda0 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:40.923475 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a651d-2939-4383-b7df-007a3ca1cda0 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:40.924209 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34a651d-2939-4383-b7df-007a3ca1cda0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:21:40.924516 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34a651d-2939-4383-b7df-007a3ca1cda0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:40.925969 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34a651d-2939-4383-b7df-007a3ca1cda0 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:40.939446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 645/1257] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:21:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:21:40.945220 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67dd398-3639-4f05-be38-045a11419833 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:41.053938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67dd398-3639-4f05-be38-045a11419833 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:41.088765 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17adb336-cfbf-49c4-b8b2-5fda20281e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:41.089605 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17adb336-cfbf-49c4-b8b2-5fda20281e53 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:41.089889 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17adb336-cfbf-49c4-b8b2-5fda20281e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:41.090165 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17adb336-cfbf-49c4-b8b2-5fda20281e53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:41.097533 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17adb336-cfbf-49c4-b8b2-5fda20281e53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:41.099045 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17adb336-cfbf-49c4-b8b2-5fda20281e53 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:41.113838 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-17adb336-cfbf-49c4-b8b2-5fda20281e53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:21:41.113838 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:21:41.113838 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:21:41.113838 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:21:41.113838 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:21:41.113838 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:21:41.113838 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:21:41.113838 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:21:41.113838 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:21:41.113838 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:21:41.113838 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:21:41.113838 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:21:41.113838 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:21:41.113838 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:21:41.113838 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:21:41.115070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:21:41.115070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:21:41.115070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:21:41.115070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:21:41.115070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:21:41.115070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:21:41.115070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:21:41.115070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:21:41.115070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:21:41.115070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:21:41.115070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:21:41.115070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:21:41.115070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:21:41.115070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:21:41.115070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:21:41.115070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:21:41.115070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:21:41.116173 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:21:41.116173 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:21:41.116173 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:21:41.116173 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:21:41.116173 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:21:41.116173 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:21:41.116173 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:21:41.116173 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:21:41.116173 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 613/1258] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:21:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:21:41.121100 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a651d-2939-4383-b7df-007a3ca1cda0 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:41.221382 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a651d-2939-4383-b7df-007a3ca1cda0 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:41.256942 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31036e62-0cdc-4383-8c91-f0dc86fac970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:41.257604 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31036e62-0cdc-4383-8c91-f0dc86fac970 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:41.257886 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31036e62-0cdc-4383-8c91-f0dc86fac970 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:41.258162 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31036e62-0cdc-4383-8c91-f0dc86fac970 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:41.258972 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31036e62-0cdc-4383-8c91-f0dc86fac970 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:21:41.259302 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31036e62-0cdc-4383-8c91-f0dc86fac970 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:41.260743 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31036e62-0cdc-4383-8c91-f0dc86fac970 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:41.273372 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 646/1259] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 21:21:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:21:41.279815 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17adb336-cfbf-49c4-b8b2-5fda20281e53 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:41.308040 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:41.385404 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17adb336-cfbf-49c4-b8b2-5fda20281e53 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:41.420606 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2827a-2838-498e-bc52-ba2a2c1591cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:41.421264 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2827a-2838-498e-bc52-ba2a2c1591cf None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:41.421546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2827a-2838-498e-bc52-ba2a2c1591cf None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:41.421813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2827a-2838-498e-bc52-ba2a2c1591cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:41.422675 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c2827a-2838-498e-bc52-ba2a2c1591cf None admin] RBAC: Adding query filter params (role.id=2b714f2e5c6c469f8fa037bda1ed16f6, scope.project.id=c4940befa9c4405699c72deedc8a2127, user.id=f6143bddc05a46ea972709576c7ad95f) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:21:41.422969 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c2827a-2838-498e-bc52-ba2a2c1591cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:41.424417 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c2827a-2838-498e-bc52-ba2a2c1591cf None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:41.430218 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 614/1260] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:21:41 2022] GET /identity/v3/role_assignments?user.id=f6143bddc05a46ea972709576c7ad95f&scope.project.id=c4940befa9c4405699c72deedc8a2127&role.id=2b714f2e5c6c469f8fa037bda1ed16f6 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:21:43.698774 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67e0df-4547-4896-82b0-1ac2248b74dc None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:43.699345 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67e0df-4547-4896-82b0-1ac2248b74dc None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:43.699345 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67e0df-4547-4896-82b0-1ac2248b74dc None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:43.700819 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 647/1261] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:21:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:21:43.707150 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd8aa9-5dd2-442b-ae37-b65fa8f210e8 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:43.707708 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd8aa9-5dd2-442b-ae37-b65fa8f210e8 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:43.707708 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd8aa9-5dd2-442b-ae37-b65fa8f210e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:43.727197 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-40dd8aa9-5dd2-442b-ae37-b65fa8f210e8 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:21:43.728452 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-40dd8aa9-5dd2-442b-ae37-b65fa8f210e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:21:43.803664 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 615/1262] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:21:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:21:43.826535 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67e0df-4547-4896-82b0-1ac2248b74dc None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:43.838196 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:43.939793 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67e0df-4547-4896-82b0-1ac2248b74dc None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:43.973869 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728c33bf-6f97-43d3-9884-aabef0b2812a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:43.974623 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728c33bf-6f97-43d3-9884-aabef0b2812a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:43.974901 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728c33bf-6f97-43d3-9884-aabef0b2812a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:43.975173 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728c33bf-6f97-43d3-9884-aabef0b2812a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:43.988732 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728c33bf-6f97-43d3-9884-aabef0b2812a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:21:43.996446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-728c33bf-6f97-43d3-9884-aabef0b2812a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:21:43.996446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:21:43.996446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:21:43.996446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:21:43.996446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:21:43.996446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:21:43.996446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:21:43.996446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:21:43.996446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:21:43.996446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:21:43.996446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:21:43.996446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:21:43.996446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:21:43.996446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:21:43.996446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:21:43.996446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:21:43.997782 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:21:43.997782 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:21:43.997782 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:21:43.997782 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:21:43.997782 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:21:43.997782 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:21:43.997782 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:21:43.997782 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:21:43.997782 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:21:43.997782 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:21:43.997782 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:21:43.997782 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:21:43.997782 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:21:43.997782 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:21:43.997782 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:21:43.997782 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:21:43.999011 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:21:43.999011 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:21:43.999011 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:21:43.999011 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:21:43.999011 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:21:43.999011 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:21:43.999011 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:21:43.999011 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:21:43.999011 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:21:43.999011 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:21:43.999011 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:21:43.999011 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:21:43.999011 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 648/1263] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 21:21:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:21:44.004351 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd8aa9-5dd2-442b-ae37-b65fa8f210e8 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:44.025105 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:44.111364 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd8aa9-5dd2-442b-ae37-b65fa8f210e8 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:44.147601 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de60f8b-def3-44e5-893d-3c991ca4a697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:44.148277 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de60f8b-def3-44e5-893d-3c991ca4a697 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:44.148561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de60f8b-def3-44e5-893d-3c991ca4a697 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:44.148837 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de60f8b-def3-44e5-893d-3c991ca4a697 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:44.149593 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de60f8b-def3-44e5-893d-3c991ca4a697 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:21:44.149901 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de60f8b-def3-44e5-893d-3c991ca4a697 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:44.151321 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de60f8b-def3-44e5-893d-3c991ca4a697 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:44.160067 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 616/1264] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:21:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:21:44.166138 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728c33bf-6f97-43d3-9884-aabef0b2812a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:44.236963 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:44.274656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728c33bf-6f97-43d3-9884-aabef0b2812a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:44.311870 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9f820-62a6-4c9a-ac76-466575bcd581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:44.312680 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9f820-62a6-4c9a-ac76-466575bcd581 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:44.312961 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9f820-62a6-4c9a-ac76-466575bcd581 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:44.313239 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9f820-62a6-4c9a-ac76-466575bcd581 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:44.326101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd9f820-62a6-4c9a-ac76-466575bcd581 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:44.327612 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd9f820-62a6-4c9a-ac76-466575bcd581 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:44.346044 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-7cd9f820-62a6-4c9a-ac76-466575bcd581 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:21:44.346044 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:21:44.346044 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:21:44.346044 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:21:44.346044 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:21:44.346044 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:21:44.346044 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:21:44.346044 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:21:44.346044 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:21:44.346044 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:21:44.346044 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:21:44.346044 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:21:44.346044 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:21:44.346044 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:21:44.346044 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:21:44.347139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:21:44.347139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:21:44.347139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:21:44.347139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:21:44.347139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:21:44.347139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:21:44.347139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:21:44.347139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:21:44.347139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:21:44.347139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:21:44.347139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:21:44.347139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:21:44.347139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:21:44.347139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:21:44.347139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:21:44.347139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:21:44.347139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:21:44.348152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:21:44.348152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:21:44.348152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:21:44.348152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:21:44.348152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 649/1265] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:21:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:21:44.353283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de60f8b-def3-44e5-893d-3c991ca4a697 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:44.439026 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:44.459408 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de60f8b-def3-44e5-893d-3c991ca4a697 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:44.495409 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26882241-9416-4ea5-8270-a20edb1d7225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:44.496018 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26882241-9416-4ea5-8270-a20edb1d7225 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:44.496307 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26882241-9416-4ea5-8270-a20edb1d7225 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:44.496579 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26882241-9416-4ea5-8270-a20edb1d7225 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:44.497474 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26882241-9416-4ea5-8270-a20edb1d7225 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:21:44.497894 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26882241-9416-4ea5-8270-a20edb1d7225 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:44.499317 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26882241-9416-4ea5-8270-a20edb1d7225 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:44.511322 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 617/1266] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:21:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:21:44.517489 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9f820-62a6-4c9a-ac76-466575bcd581 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:44.625008 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd9f820-62a6-4c9a-ac76-466575bcd581 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:44.646895 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:44.663431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c618ab-2639-4f20-8df5-44e8bc056868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:44.664345 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c618ab-2639-4f20-8df5-44e8bc056868 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:44.664631 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c618ab-2639-4f20-8df5-44e8bc056868 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:44.664933 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c618ab-2639-4f20-8df5-44e8bc056868 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:44.674322 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c618ab-2639-4f20-8df5-44e8bc056868 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:44.675810 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c618ab-2639-4f20-8df5-44e8bc056868 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:44.691261 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-08c618ab-2639-4f20-8df5-44e8bc056868 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 21:21:44.691261 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:21:44.691261 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:21:44.691261 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:21:44.691261 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:21:44.691261 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:21:44.691261 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:21:44.691261 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:21:44.691261 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:21:44.691261 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:21:44.691261 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:21:44.691261 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:21:44.691261 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:21:44.691261 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:21:44.691261 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:21:44.691261 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:21:44.692396 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:21:44.692396 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:21:44.692396 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:21:44.692396 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:21:44.692396 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:21:44.692396 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:21:44.692396 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:21:44.692396 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:21:44.692396 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:21:44.692396 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:21:44.692396 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:21:44.692396 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:21:44.692396 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:21:44.692396 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:21:44.692396 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:21:44.692396 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:21:44.692396 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:21:44.692396 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:21:44.693794 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:21:44.693794 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:21:44.693794 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:21:44.693794 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:21:44.693794 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:21:44.693794 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:21:44.693794 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:21:44.693794 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:21:44.693794 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 21:21:44.693794 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:21:44.693794 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 650/1267] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:21:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:21:44.698521 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26882241-9416-4ea5-8270-a20edb1d7225 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:44.802734 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26882241-9416-4ea5-8270-a20edb1d7225 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:44.838222 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d65f6a-6e4f-45bd-9f34-a7f5d138f720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:44.838931 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d65f6a-6e4f-45bd-9f34-a7f5d138f720 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:44.839214 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d65f6a-6e4f-45bd-9f34-a7f5d138f720 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:44.839487 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d65f6a-6e4f-45bd-9f34-a7f5d138f720 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:44.840330 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d65f6a-6e4f-45bd-9f34-a7f5d138f720 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:21:44.840680 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d65f6a-6e4f-45bd-9f34-a7f5d138f720 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:44.842149 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d65f6a-6e4f-45bd-9f34-a7f5d138f720 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:44.852098 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:44.862822 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 618/1268] 10.222.0.65 () {56 vars in 1143 bytes} [Thu May 19 21:21:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:21:44.868810 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c618ab-2639-4f20-8df5-44e8bc056868 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:44.975845 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c618ab-2639-4f20-8df5-44e8bc056868 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:45.011343 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798747a-1384-48e6-beaf-67339d094742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:45.012142 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798747a-1384-48e6-beaf-67339d094742 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:45.012499 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798747a-1384-48e6-beaf-67339d094742 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:45.012779 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798747a-1384-48e6-beaf-67339d094742 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:45.024022 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d798747a-1384-48e6-beaf-67339d094742 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:45.026076 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d798747a-1384-48e6-beaf-67339d094742 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:45.053683 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-d798747a-1384-48e6-beaf-67339d094742 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:21:45.053683 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:21:45.053683 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:21:45.053683 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:21:45.053683 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:21:45.053683 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:21:45.053683 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:21:45.053683 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:21:45.053683 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:21:45.053683 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:21:45.053683 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:21:45.053683 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:21:45.053683 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:21:45.053683 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:21:45.053683 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:21:45.054807 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:21:45.054807 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:21:45.054807 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:21:45.054807 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:21:45.054807 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:21:45.054807 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:21:45.054807 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:21:45.054807 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:21:45.054807 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:21:45.054807 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:21:45.054807 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:21:45.054807 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:21:45.054807 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:21:45.054807 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:21:45.054807 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:21:45.054807 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:21:45.054807 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:21:45.055773 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:21:45.055773 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:21:45.055773 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:21:45.055773 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:21:45.055773 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 651/1269] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:21:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:21:45.062899 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d65f6a-6e4f-45bd-9f34-a7f5d138f720 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:45.179166 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d65f6a-6e4f-45bd-9f34-a7f5d138f720 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:45.215887 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf9efbe-9697-41f8-83b8-0e0cb111a13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:45.216851 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9efbe-9697-41f8-83b8-0e0cb111a13a None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:45.217132 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9efbe-9697-41f8-83b8-0e0cb111a13a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:45.217411 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9efbe-9697-41f8-83b8-0e0cb111a13a None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:45.218130 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf9efbe-9697-41f8-83b8-0e0cb111a13a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:21:45.218471 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf9efbe-9697-41f8-83b8-0e0cb111a13a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:45.219867 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf9efbe-9697-41f8-83b8-0e0cb111a13a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:45.232406 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 619/1270] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:21:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:21:45.238568 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798747a-1384-48e6-beaf-67339d094742 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:45.275384 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:45.347653 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798747a-1384-48e6-beaf-67339d094742 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:45.392812 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acecbbb0-1afe-48e0-ae32-6bc994334ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:45.393634 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecbbb0-1afe-48e0-ae32-6bc994334ad3 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:45.393925 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecbbb0-1afe-48e0-ae32-6bc994334ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:45.394202 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecbbb0-1afe-48e0-ae32-6bc994334ad3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:45.402788 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acecbbb0-1afe-48e0-ae32-6bc994334ad3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:45.404607 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acecbbb0-1afe-48e0-ae32-6bc994334ad3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:45.419640 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-acecbbb0-1afe-48e0-ae32-6bc994334ad3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:21:45.419640 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:21:45.419640 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:21:45.419640 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:21:45.419640 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:21:45.419640 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:21:45.419640 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:21:45.419640 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:21:45.419640 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:21:45.419640 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:21:45.419640 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:21:45.419640 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:21:45.419640 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:21:45.419640 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:21:45.419640 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:21:45.420783 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:21:45.420783 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:21:45.420783 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:21:45.420783 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:21:45.420783 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:21:45.420783 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:21:45.420783 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:21:45.420783 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:21:45.420783 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:21:45.420783 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:21:45.420783 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:21:45.420783 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:21:45.420783 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:21:45.420783 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:21:45.420783 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:21:45.420783 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:21:45.420783 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:21:45.421926 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:21:45.421926 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:21:45.421926 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:21:45.421926 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:21:45.421926 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:21:45.421926 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:21:45.421926 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:21:45.421926 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:21:45.421926 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 652/1271] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:21:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:21:45.426981 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf9efbe-9697-41f8-83b8-0e0cb111a13a None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:45.491942 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:45.537326 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf9efbe-9697-41f8-83b8-0e0cb111a13a None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:45.572895 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d70d350-33b3-4780-907f-083d134e7cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:45.573542 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70d350-33b3-4780-907f-083d134e7cba None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:45.573810 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70d350-33b3-4780-907f-083d134e7cba None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:45.574084 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70d350-33b3-4780-907f-083d134e7cba None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:45.575367 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d70d350-33b3-4780-907f-083d134e7cba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:21:45.575701 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d70d350-33b3-4780-907f-083d134e7cba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:45.577148 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d70d350-33b3-4780-907f-083d134e7cba None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:45.591151 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 620/1272] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 21:21:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:21:45.597599 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acecbbb0-1afe-48e0-ae32-6bc994334ad3 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:45.699689 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:45.710025 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acecbbb0-1afe-48e0-ae32-6bc994334ad3 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:45.752926 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432dbe13-b064-4446-8261-3a8e7f7cf7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:45.753831 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432dbe13-b064-4446-8261-3a8e7f7cf7c4 None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:45.754114 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432dbe13-b064-4446-8261-3a8e7f7cf7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:45.754420 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432dbe13-b064-4446-8261-3a8e7f7cf7c4 None admin] PATH_INFO: `/v3/projects/c4940befa9c4405699c72deedc8a2127/users/f6143bddc05a46ea972709576c7ad95f/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:45.756491 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432dbe13-b064-4446-8261-3a8e7f7cf7c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4940befa9c4405699c72deedc8a2127, user_id=f6143bddc05a46ea972709576c7ad95f, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:45.757935 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432dbe13-b064-4446-8261-3a8e7f7cf7c4 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:45.772696 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 653/1273] 10.222.0.65 () {58 vars in 1439 bytes} [Thu May 19 21:21:45 2022] PUT /identity/v3/projects/c4940befa9c4405699c72deedc8a2127/users/f6143bddc05a46ea972709576c7ad95f/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:21:47.962531 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7a30e-97c2-444f-8c04-7349d3f1ba42 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:47.963084 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7a30e-97c2-444f-8c04-7349d3f1ba42 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:47.963084 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7a30e-97c2-444f-8c04-7349d3f1ba42 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:47.964464 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 621/1274] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:21:47.969942 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8d6ea-f824-4718-ab2e-b6d72fb1ff28 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:47.970447 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8d6ea-f824-4718-ab2e-b6d72fb1ff28 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:47.970553 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8d6ea-f824-4718-ab2e-b6d72fb1ff28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:47.988973 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-72a8d6ea-f824-4718-ab2e-b6d72fb1ff28 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:21:47.990184 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-72a8d6ea-f824-4718-ab2e-b6d72fb1ff28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:21:48.062514 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 654/1275] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:21:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:21:48.086917 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b7a30e-97c2-444f-8c04-7349d3f1ba42 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:48.194283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b7a30e-97c2-444f-8c04-7349d3f1ba42 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:48.212199 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:48.232667 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9460e-9ebf-4492-a345-21fb425ef6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:48.233518 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9460e-9ebf-4492-a345-21fb425ef6cd None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:48.233794 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9460e-9ebf-4492-a345-21fb425ef6cd None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:48.234062 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9460e-9ebf-4492-a345-21fb425ef6cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:48.245086 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f9460e-9ebf-4492-a345-21fb425ef6cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 21:21:48.252938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-73f9460e-9ebf-4492-a345-21fb425ef6cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:21:48.252938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:21:48.252938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:21:48.252938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:21:48.252938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:21:48.252938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:21:48.252938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:21:48.252938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:21:48.252938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:21:48.252938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:21:48.252938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:21:48.252938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:21:48.252938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 21:21:48.252938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 21:21:48.252938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 21:21:48.252938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 21:21:48.254017 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 21:21:48.254017 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 21:21:48.254017 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 21:21:48.254017 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 21:21:48.254017 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:21:48.254017 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:21:48.254017 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:21:48.254017 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:21:48.254017 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:21:48.254017 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:21:48.254017 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:21:48.254017 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:21:48.254017 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:21:48.254017 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:21:48.254017 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:21:48.254017 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:21:48.255090 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:21:48.255090 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:21:48.255090 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:21:48.255090 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:21:48.255090 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 21:21:48.255090 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 21:21:48.255090 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 21:21:48.255090 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 21:21:48.255090 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 21:21:48.255090 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 21:21:48.255090 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 21:21:48.255090 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:21:48.255090 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 622/1276] 10.222.0.65 () {56 vars in 1096 bytes} [Thu May 19 21:21:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:21:48.260308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a8d6ea-f824-4718-ab2e-b6d72fb1ff28 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:48.345150 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:48.366277 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a8d6ea-f824-4718-ab2e-b6d72fb1ff28 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:48.401837 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440fb076-e446-4356-a853-f05c88802b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:48.402543 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440fb076-e446-4356-a853-f05c88802b8e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:48.402915 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440fb076-e446-4356-a853-f05c88802b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:48.403198 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440fb076-e446-4356-a853-f05c88802b8e None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:48.403951 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440fb076-e446-4356-a853-f05c88802b8e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:21:48.404269 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440fb076-e446-4356-a853-f05c88802b8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:48.405662 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440fb076-e446-4356-a853-f05c88802b8e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:48.414652 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 655/1277] 10.222.0.65 () {56 vars in 1099 bytes} [Thu May 19 21:21:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:21:48.420872 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9460e-9ebf-4492-a345-21fb425ef6cd None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:48.528021 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9460e-9ebf-4492-a345-21fb425ef6cd None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:48.567737 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2986ce-f6dc-4606-b58c-4390c68722fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:48.568551 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2986ce-f6dc-4606-b58c-4390c68722fd None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:48.568830 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2986ce-f6dc-4606-b58c-4390c68722fd None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:48.569097 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2986ce-f6dc-4606-b58c-4390c68722fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:48.580921 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2986ce-f6dc-4606-b58c-4390c68722fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:48.582431 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2986ce-f6dc-4606-b58c-4390c68722fd None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:48.602928 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-bb2986ce-f6dc-4606-b58c-4390c68722fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:21:48.602928 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:21:48.602928 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:21:48.602928 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:21:48.602928 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:21:48.602928 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:21:48.602928 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:21:48.602928 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:21:48.602928 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:21:48.602928 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:21:48.602928 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:21:48.602928 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:21:48.602928 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:21:48.602928 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:21:48.602928 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:21:48.604063 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:21:48.604063 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:21:48.604063 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:21:48.604063 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:21:48.604063 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:21:48.604063 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:21:48.604063 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:21:48.604063 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:21:48.604063 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:21:48.604063 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:21:48.604063 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:21:48.604063 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:21:48.604063 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:21:48.604063 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:21:48.604063 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:21:48.604063 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:21:48.604063 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:21:48.604968 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:21:48.604968 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:21:48.604968 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:21:48.604968 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:21:48.604968 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 623/1278] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:21:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:21:48.612276 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440fb076-e446-4356-a853-f05c88802b8e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:48.725839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440fb076-e446-4356-a853-f05c88802b8e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:48.763812 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5ed5a-44f4-4812-9fdd-75572f8eed62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:48.764483 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5ed5a-44f4-4812-9fdd-75572f8eed62 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:48.764777 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5ed5a-44f4-4812-9fdd-75572f8eed62 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:48.765038 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5ed5a-44f4-4812-9fdd-75572f8eed62 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:48.765762 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d5ed5a-44f4-4812-9fdd-75572f8eed62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:21:48.766068 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d5ed5a-44f4-4812-9fdd-75572f8eed62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:48.767478 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d5ed5a-44f4-4812-9fdd-75572f8eed62 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:48.780303 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 656/1279] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:21:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:21:48.786180 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2986ce-f6dc-4606-b58c-4390c68722fd None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:48.818503 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:48.893289 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2986ce-f6dc-4606-b58c-4390c68722fd None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:48.929456 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46eb8c8-6009-414a-9a76-c247a7be1340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:48.930283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46eb8c8-6009-414a-9a76-c247a7be1340 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:48.930564 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46eb8c8-6009-414a-9a76-c247a7be1340 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:48.930839 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46eb8c8-6009-414a-9a76-c247a7be1340 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:48.938778 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46eb8c8-6009-414a-9a76-c247a7be1340 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:48.940230 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46eb8c8-6009-414a-9a76-c247a7be1340 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:48.954722 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-e46eb8c8-6009-414a-9a76-c247a7be1340 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 21:21:48.954722 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:21:48.954722 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:21:48.954722 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:21:48.954722 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:21:48.954722 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:21:48.954722 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:21:48.954722 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:21:48.954722 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:21:48.954722 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:21:48.954722 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:21:48.954722 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:21:48.954722 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 21:21:48.954722 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 21:21:48.954722 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 21:21:48.954722 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 21:21:48.956404 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:21:48.956404 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:21:48.956404 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 21:21:48.956404 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:21:48.956404 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 21:21:48.956404 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 21:21:48.956404 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:21:48.956404 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:21:48.956404 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:21:48.956404 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:21:48.956404 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:21:48.956404 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:21:48.956404 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:21:48.956404 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:21:48.956404 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:21:48.956404 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:21:48.956404 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:21:48.956404 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:21:48.958232 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:21:48.958232 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:21:48.958232 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 21:21:48.958232 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 21:21:48.958232 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 21:21:48.958232 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 21:21:48.958232 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 21:21:48.958232 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 21:21:48.958232 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 21:21:48.958232 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:21:48.958232 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 624/1280] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:21:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:21:48.963491 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5ed5a-44f4-4812-9fdd-75572f8eed62 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:48.968110 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:49.069694 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5ed5a-44f4-4812-9fdd-75572f8eed62 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:49.106827 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a095c92-46c4-4f60-89ae-26cd51f97add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:49.107694 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a095c92-46c4-4f60-89ae-26cd51f97add None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:49.107977 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a095c92-46c4-4f60-89ae-26cd51f97add None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:49.108252 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a095c92-46c4-4f60-89ae-26cd51f97add None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:49.109091 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a095c92-46c4-4f60-89ae-26cd51f97add None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:21:49.109402 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a095c92-46c4-4f60-89ae-26cd51f97add None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:49.110884 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a095c92-46c4-4f60-89ae-26cd51f97add None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:49.129033 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 657/1281] 10.222.0.65 () {56 vars in 1143 bytes} [Thu May 19 21:21:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:21:49.135055 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46eb8c8-6009-414a-9a76-c247a7be1340 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:49.230964 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:49.242490 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46eb8c8-6009-414a-9a76-c247a7be1340 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:49.278798 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58125a50-d7fb-4453-8e9a-4c6c107baa99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:49.279458 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58125a50-d7fb-4453-8e9a-4c6c107baa99 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:49.279726 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58125a50-d7fb-4453-8e9a-4c6c107baa99 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:49.279999 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58125a50-d7fb-4453-8e9a-4c6c107baa99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:49.360555 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58125a50-d7fb-4453-8e9a-4c6c107baa99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:49.362041 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58125a50-d7fb-4453-8e9a-4c6c107baa99 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:49.456660 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:49.468151 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 625/1282] 10.222.0.65 () {58 vars in 1282 bytes} [Thu May 19 21:21:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 21:21:49.474160 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a095c92-46c4-4f60-89ae-26cd51f97add None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:49.554473 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:49.600129 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a095c92-46c4-4f60-89ae-26cd51f97add None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:49.636934 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d586a6-1c85-4caf-9ece-5184b8665de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:49.637882 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d586a6-1c85-4caf-9ece-5184b8665de8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:49.638266 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d586a6-1c85-4caf-9ece-5184b8665de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:49.638666 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d586a6-1c85-4caf-9ece-5184b8665de8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:49.649587 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d586a6-1c85-4caf-9ece-5184b8665de8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:49.651076 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d586a6-1c85-4caf-9ece-5184b8665de8 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:49.669477 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-62d586a6-1c85-4caf-9ece-5184b8665de8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:21:49.669477 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:21:49.669477 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:21:49.669477 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:21:49.669477 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:21:49.669477 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:21:49.669477 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:21:49.669477 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:21:49.669477 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:21:49.669477 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:21:49.669477 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:21:49.669477 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:21:49.669477 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 21:21:49.669477 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 21:21:49.669477 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 21:21:49.670564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 21:21:49.670564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:21:49.670564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:21:49.670564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:21:49.670564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:21:49.670564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:21:49.670564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:21:49.670564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:21:49.670564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:21:49.670564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:21:49.670564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:21:49.670564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:21:49.670564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:21:49.670564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:21:49.670564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:21:49.670564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:21:49.670564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:21:49.671832 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 21:21:49.671832 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 21:21:49.671832 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 21:21:49.671832 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:21:49.671832 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 658/1283] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:21:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:21:49.676588 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58125a50-d7fb-4453-8e9a-4c6c107baa99 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:49.780113 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58125a50-d7fb-4453-8e9a-4c6c107baa99 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:49.819657 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21dee20-a07f-4672-be4e-352f61300fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:49.820334 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21dee20-a07f-4672-be4e-352f61300fdd None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:49.820613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21dee20-a07f-4672-be4e-352f61300fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:49.820888 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21dee20-a07f-4672-be4e-352f61300fdd None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:49.821626 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21dee20-a07f-4672-be4e-352f61300fdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:21:49.821930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21dee20-a07f-4672-be4e-352f61300fdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:49.823456 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21dee20-a07f-4672-be4e-352f61300fdd None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:49.837035 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 626/1284] 10.222.0.65 () {56 vars in 1109 bytes} [Thu May 19 21:21:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:21:49.843775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d586a6-1c85-4caf-9ece-5184b8665de8 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:49.952622 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d586a6-1c85-4caf-9ece-5184b8665de8 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:49.966996 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:49.990603 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffbadc0-4e72-44c3-b732-3f4e55e6ffa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:49.991411 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbadc0-4e72-44c3-b732-3f4e55e6ffa3 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:49.991696 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbadc0-4e72-44c3-b732-3f4e55e6ffa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:49.991971 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbadc0-4e72-44c3-b732-3f4e55e6ffa3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:49.999586 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffbadc0-4e72-44c3-b732-3f4e55e6ffa3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:50.001046 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffbadc0-4e72-44c3-b732-3f4e55e6ffa3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:50.015149 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-9ffbadc0-4e72-44c3-b732-3f4e55e6ffa3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:21:50.015149 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:21:50.015149 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:21:50.015149 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:21:50.015149 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:21:50.015149 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:21:50.015149 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:21:50.015149 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:21:50.015149 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:21:50.015149 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:21:50.015149 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:21:50.015149 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:21:50.015149 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 21:21:50.015149 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 21:21:50.015149 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 21:21:50.016187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 21:21:50.016187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:21:50.016187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:21:50.016187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:21:50.016187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:21:50.016187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:21:50.016187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:21:50.016187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:21:50.016187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 19 21:21:50.016187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:21:50.016187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 19 21:21:50.016187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:21:50.016187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:21:50.016187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:21:50.016187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 19 21:21:50.016187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:21:50.016187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 19 21:21:50.017126 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 21:21:50.017126 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 21:21:50.017126 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 21:21:50.017126 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 21:21:50.017126 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 21:21:50.017126 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 21:21:50.017126 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 21:21:50.017126 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 19 21:21:50.017126 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 659/1285] 10.222.0.65 () {56 vars in 1111 bytes} [Thu May 19 21:21:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:21:50.022501 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21dee20-a07f-4672-be4e-352f61300fdd None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:50.076589 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:50.138049 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21dee20-a07f-4672-be4e-352f61300fdd None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:50.177981 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd8756-f0a6-4908-b281-be935bc75b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:50.178687 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd8756-f0a6-4908-b281-be935bc75b8d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:50.178967 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd8756-f0a6-4908-b281-be935bc75b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:50.179384 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd8756-f0a6-4908-b281-be935bc75b8d None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:50.180174 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd8756-f0a6-4908-b281-be935bc75b8d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:21:50.180484 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd8756-f0a6-4908-b281-be935bc75b8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:50.181926 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd8756-f0a6-4908-b281-be935bc75b8d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:50.194776 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 627/1286] 10.222.0.65 () {56 vars in 1148 bytes} [Thu May 19 21:21:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:21:50.201428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffbadc0-4e72-44c3-b732-3f4e55e6ffa3 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:50.309953 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffbadc0-4e72-44c3-b732-3f4e55e6ffa3 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:50.349080 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554f4df1-8035-4d24-95d7-3963838680b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:50.349751 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f4df1-8035-4d24-95d7-3963838680b4 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:50.350031 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f4df1-8035-4d24-95d7-3963838680b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:50.350316 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f4df1-8035-4d24-95d7-3963838680b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:50.351185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554f4df1-8035-4d24-95d7-3963838680b4 None admin] RBAC: Adding query filter params (role.id=2b714f2e5c6c469f8fa037bda1ed16f6, scope.project.id=c4940befa9c4405699c72deedc8a2127, user.id=f6143bddc05a46ea972709576c7ad95f) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:21:50.351493 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554f4df1-8035-4d24-95d7-3963838680b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:50.352985 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554f4df1-8035-4d24-95d7-3963838680b4 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:50.360537 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 660/1287] 10.222.0.65 () {56 vars in 1374 bytes} [Thu May 19 21:21:50 2022] GET /identity/v3/role_assignments?user.id=f6143bddc05a46ea972709576c7ad95f&scope.project.id=c4940befa9c4405699c72deedc8a2127&role.id=2b714f2e5c6c469f8fa037bda1ed16f6 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:21:52.586764 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46306411-1966-431d-ad1a-14e420f9fdd2 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:52.587554 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46306411-1966-431d-ad1a-14e420f9fdd2 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:52.587554 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46306411-1966-431d-ad1a-14e420f9fdd2 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:52.588646 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 628/1288] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:21:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:21:52.596791 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd341992-7953-4b4d-85c7-59c01b2265f3 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:52.597381 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd341992-7953-4b4d-85c7-59c01b2265f3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:52.598105 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd341992-7953-4b4d-85c7-59c01b2265f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:52.619409 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-cd341992-7953-4b4d-85c7-59c01b2265f3 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:21:52.620644 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cd341992-7953-4b4d-85c7-59c01b2265f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:21:52.653655 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:52.698275 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 661/1289] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:21:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:21:52.721691 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46306411-1966-431d-ad1a-14e420f9fdd2 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:52.830705 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46306411-1966-431d-ad1a-14e420f9fdd2 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:52.841456 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:52.868731 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de4847-4175-4584-8162-fd26896e2a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:52.869552 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de4847-4175-4584-8162-fd26896e2a51 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:52.869837 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de4847-4175-4584-8162-fd26896e2a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:52.870109 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de4847-4175-4584-8162-fd26896e2a51 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:52.870795 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de4847-4175-4584-8162-fd26896e2a51 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:52.872823 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de4847-4175-4584-8162-fd26896e2a51 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:52.884992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-f9de4847-4175-4584-8162-fd26896e2a51 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 21:21:52.884992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:21:52.884992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:21:52.884992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:21:52.884992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:21:52.884992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:21:52.884992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:21:52.884992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:21:52.884992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 21:21:52.884992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:21:52.884992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:21:52.884992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:21:52.884992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:21:52.884992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:21:52.884992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:21:52.884992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:21:52.884992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:21:52.886398 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:21:52.886398 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:21:52.886398 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:21:52.886398 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:21:52.886398 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:21:52.886398 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:21:52.886398 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:21:52.886398 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:21:52.886398 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:21:52.886398 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:21:52.886398 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:21:52.886398 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:21:52.886398 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:21:52.886398 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:21:52.886398 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:21:52.886398 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:21:52.887455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:21:52.887455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:21:52.887455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:21:52.887455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:21:52.887455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:21:52.887455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:21:52.887455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:21:52.887455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:21:52.887455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:21:52.887455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:21:52.887455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:21:52.887455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:21:52.887455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 21:21:52.887455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:21:52.887455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 629/1290] 10.222.0.65 () {56 vars in 1117 bytes} [Thu May 19 21:21:52 2022] GET /identity/v3/services/placement => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:21:52.892979 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd341992-7953-4b4d-85c7-59c01b2265f3 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:53.005022 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd341992-7953-4b4d-85c7-59c01b2265f3 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:53.040561 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ab00a-d2ae-4951-938d-0030a7462a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:53.041226 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ab00a-d2ae-4951-938d-0030a7462a97 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:53.041507 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ab00a-d2ae-4951-938d-0030a7462a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:53.041784 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ab00a-d2ae-4951-938d-0030a7462a97 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:53.042525 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ab00a-d2ae-4951-938d-0030a7462a97 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:21:53.042828 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ab00a-d2ae-4951-938d-0030a7462a97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:53.044267 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ab00a-d2ae-4951-938d-0030a7462a97 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:53.049368 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 662/1291] 10.222.0.65 () {56 vars in 1116 bytes} [Thu May 19 21:21:52 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:21:53.055563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de4847-4175-4584-8162-fd26896e2a51 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:53.161032 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de4847-4175-4584-8162-fd26896e2a51 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:53.196229 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe1f04c-ab4e-42b4-a548-6758197335bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:53.196893 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe1f04c-ab4e-42b4-a548-6758197335bf None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:53.197173 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe1f04c-ab4e-42b4-a548-6758197335bf None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:53.197447 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe1f04c-ab4e-42b4-a548-6758197335bf None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:53.198156 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe1f04c-ab4e-42b4-a548-6758197335bf None admin] RBAC: Adding query filter params (type=placement) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:21:53.198489 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe1f04c-ab4e-42b4-a548-6758197335bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:53.200055 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe1f04c-ab4e-42b4-a548-6758197335bf None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:53.204696 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 630/1292] 10.222.0.65 () {56 vars in 1116 bytes} [Thu May 19 21:21:53 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:21:55.438270 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420da5de-a1bd-4e84-8894-7636b4fac4a7 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:55.438835 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420da5de-a1bd-4e84-8894-7636b4fac4a7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:55.438835 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420da5de-a1bd-4e84-8894-7636b4fac4a7 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:55.440279 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 663/1293] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:21:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:21:55.445699 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a44c13-3052-4848-ad8b-dcedbe68ed78 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:55.446154 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a44c13-3052-4848-ad8b-dcedbe68ed78 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:55.446269 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a44c13-3052-4848-ad8b-dcedbe68ed78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:55.464632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-c0a44c13-3052-4848-ad8b-dcedbe68ed78 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:21:55.465841 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c0a44c13-3052-4848-ad8b-dcedbe68ed78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:21:55.496634 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:55.536507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 631/1294] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:21:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:21:55.559469 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420da5de-a1bd-4e84-8894-7636b4fac4a7 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:55.595904 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:55.694128 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420da5de-a1bd-4e84-8894-7636b4fac4a7 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:55.743545 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f522de7a-6efd-4bb9-a116-805303d1775f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:55.744225 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522de7a-6efd-4bb9-a116-805303d1775f None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:55.744501 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522de7a-6efd-4bb9-a116-805303d1775f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:55.744778 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522de7a-6efd-4bb9-a116-805303d1775f None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:55.745514 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f522de7a-6efd-4bb9-a116-805303d1775f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:55.747092 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f522de7a-6efd-4bb9-a116-805303d1775f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:55.763747 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 664/1295] 10.222.0.65 () {60 vars in 1141 bytes} [Thu May 19 21:21:55 2022] POST /identity/v3/services => generated 246 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:21:57.970637 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e09d76-6351-4c5f-ad90-034ae0657e72 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:57.971197 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e09d76-6351-4c5f-ad90-034ae0657e72 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:57.971197 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e09d76-6351-4c5f-ad90-034ae0657e72 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:57.972565 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 632/1296] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:21:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:21:57.978864 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335560e0-41c7-4475-af1e-603553539a6e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:57.979409 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335560e0-41c7-4475-af1e-603553539a6e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:57.979511 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335560e0-41c7-4475-af1e-603553539a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:57.998704 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-335560e0-41c7-4475-af1e-603553539a6e None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:21:57.999885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-335560e0-41c7-4475-af1e-603553539a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:21:58.075388 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 665/1297] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:21:57 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:21:58.100416 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e09d76-6351-4c5f-ad90-034ae0657e72 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:58.208304 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e09d76-6351-4c5f-ad90-034ae0657e72 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:58.243810 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db504e8e-97d3-4e90-819e-5ae4a013bb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:58.244612 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db504e8e-97d3-4e90-819e-5ae4a013bb4a None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:58.244885 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db504e8e-97d3-4e90-819e-5ae4a013bb4a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:58.245144 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db504e8e-97d3-4e90-819e-5ae4a013bb4a None admin] PATH_INFO: `/v3/services/placement` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:58.245792 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db504e8e-97d3-4e90-819e-5ae4a013bb4a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:58.247336 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db504e8e-97d3-4e90-819e-5ae4a013bb4a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:58.259288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-db504e8e-97d3-4e90-819e-5ae4a013bb4a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 21:21:58.259288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:21:58.259288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:21:58.259288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:21:58.259288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:21:58.259288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:21:58.259288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:21:58.259288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:21:58.259288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 21:21:58.259288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:21:58.259288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:21:58.259288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:21:58.259288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:21:58.259288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:21:58.259288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:21:58.259288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:21:58.259288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:21:58.260422 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:21:58.260422 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:21:58.260422 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:21:58.260422 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:21:58.260422 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:21:58.260422 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:21:58.260422 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:21:58.260422 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:21:58.260422 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:21:58.260422 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:21:58.260422 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:21:58.260422 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:21:58.260422 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:21:58.260422 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:21:58.260422 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:21:58.260422 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:21:58.261472 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:21:58.261472 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:21:58.261472 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:21:58.261472 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:21:58.261472 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:21:58.261472 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:21:58.261472 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:21:58.261472 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:21:58.261472 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:21:58.261472 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:21:58.261472 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:21:58.261472 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:21:58.261472 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 21:21:58.261472 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:21:58.266035 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 633/1298] 10.222.0.65 () {56 vars in 1117 bytes} [Thu May 19 21:21:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:21:58.267172 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335560e0-41c7-4475-af1e-603553539a6e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:58.285747 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:58.374728 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335560e0-41c7-4475-af1e-603553539a6e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:58.411591 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8179ff-275d-4381-baa2-4742f7aba7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:58.412269 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8179ff-275d-4381-baa2-4742f7aba7d3 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:58.412558 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8179ff-275d-4381-baa2-4742f7aba7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:58.412822 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8179ff-275d-4381-baa2-4742f7aba7d3 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:58.413529 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8179ff-275d-4381-baa2-4742f7aba7d3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:21:58.413832 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8179ff-275d-4381-baa2-4742f7aba7d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:58.415277 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8179ff-275d-4381-baa2-4742f7aba7d3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:58.421056 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 666/1299] 10.222.0.65 () {56 vars in 1116 bytes} [Thu May 19 21:21:58 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:21:58.427315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db504e8e-97d3-4e90-819e-5ae4a013bb4a None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:58.462632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:58.542165 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db504e8e-97d3-4e90-819e-5ae4a013bb4a None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:58.579515 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1fa6e5-a23f-434a-8fe0-c37a85f9b967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:58.580364 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1fa6e5-a23f-434a-8fe0-c37a85f9b967 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:58.580709 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1fa6e5-a23f-434a-8fe0-c37a85f9b967 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:58.580992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1fa6e5-a23f-434a-8fe0-c37a85f9b967 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:58.581792 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1fa6e5-a23f-434a-8fe0-c37a85f9b967 None admin] RBAC: Adding query filter params (interface=public, service_id=50a48f23b2684baeb997a4447b414223, region_id=RegionOne) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:21:58.582104 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1fa6e5-a23f-434a-8fe0-c37a85f9b967 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:58.583536 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1fa6e5-a23f-434a-8fe0-c37a85f9b967 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:58.589043 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 634/1300] 10.222.0.65 () {56 vars in 1251 bytes} [Thu May 19 21:21:58 2022] GET /identity/v3/endpoints?service_id=50a48f23b2684baeb997a4447b414223&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:21:58.595648 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8179ff-275d-4381-baa2-4742f7aba7d3 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:21:58.684613 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:21:58.708258 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8179ff-275d-4381-baa2-4742f7aba7d3 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:21:58.757925 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec2942-13b3-4cdc-bf84-8f44343ff620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:21:58.758664 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec2942-13b3-4cdc-bf84-8f44343ff620 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:21:58.758947 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec2942-13b3-4cdc-bf84-8f44343ff620 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:21:58.759222 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec2942-13b3-4cdc-bf84-8f44343ff620 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:21:58.759929 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfec2942-13b3-4cdc-bf84-8f44343ff620 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:21:58.761332 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfec2942-13b3-4cdc-bf84-8f44343ff620 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:21:58.769909 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 667/1301] 10.222.0.65 () {56 vars in 1087 bytes} [Thu May 19 21:21:58 2022] GET /identity/v3/services => generated 2191 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:22:01.002085 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e8832-9926-4b24-ae2d-bd581b17e83e None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:22:01.002755 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e8832-9926-4b24-ae2d-bd581b17e83e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:22:01.002755 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e8832-9926-4b24-ae2d-bd581b17e83e None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:22:01.004054 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 635/1302] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:22:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:22:01.009407 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169dc91-9559-42e0-b32c-197705e47fee None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:22:01.009862 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169dc91-9559-42e0-b32c-197705e47fee None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:22:01.009959 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169dc91-9559-42e0-b32c-197705e47fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:22:01.027444 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c169dc91-9559-42e0-b32c-197705e47fee None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:22:01.028632 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c169dc91-9559-42e0-b32c-197705e47fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:22:01.098925 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 668/1303] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:22:01 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 21:22:01.122137 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993e8832-9926-4b24-ae2d-bd581b17e83e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:22:01.225938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:22:01.234808 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993e8832-9926-4b24-ae2d-bd581b17e83e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:22:01.273523 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711f9e8-3053-46f0-a46f-51ac3723c175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:22:01.274384 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711f9e8-3053-46f0-a46f-51ac3723c175 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:22:01.274666 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711f9e8-3053-46f0-a46f-51ac3723c175 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:22:01.274941 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711f9e8-3053-46f0-a46f-51ac3723c175 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:22:01.275601 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9711f9e8-3053-46f0-a46f-51ac3723c175 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:22:01.276987 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9711f9e8-3053-46f0-a46f-51ac3723c175 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:22:01.290140 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-9711f9e8-3053-46f0-a46f-51ac3723c175 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 21:22:01.290140 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:22:01.290140 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 21:22:01.290140 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 21:22:01.290140 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 21:22:01.290140 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 21:22:01.290140 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 21:22:01.290140 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:22:01.290140 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 21:22:01.290140 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:22:01.290140 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 21:22:01.290140 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:22:01.290140 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 21:22:01.290140 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 21:22:01.290140 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 21:22:01.290140 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 21:22:01.290140 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 21:22:01.291439 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 21:22:01.291439 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 21:22:01.291439 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 21:22:01.291439 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 21:22:01.291439 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 21:22:01.291439 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 21:22:01.291439 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 21:22:01.291439 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 21:22:01.291439 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 21:22:01.291439 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 21:22:01.291439 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 21:22:01.291439 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 21:22:01.291439 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 21:22:01.291439 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 21:22:01.291439 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 21:22:01.291439 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( May 19 21:22:01.292382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 21:22:01.292382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( May 19 21:22:01.292382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 21:22:01.292382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() May 19 21:22:01.292382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 21:22:01.292382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 21:22:01.292382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 21:22:01.292382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() May 19 21:22:01.292382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 21:22:01.292382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( May 19 21:22:01.292382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 21:22:01.292382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 21:22:01.292382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 21:22:01.292382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application May 19 21:22:01.292382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 636/1304] 10.222.0.65 () {56 vars in 1117 bytes} [Thu May 19 21:22:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 21:22:01.298114 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c169dc91-9559-42e0-b32c-197705e47fee None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:22:01.381423 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:22:01.409012 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c169dc91-9559-42e0-b32c-197705e47fee None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:22:01.448341 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91492a1-a005-480d-a11f-6ad74c3a8cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:22:01.449060 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91492a1-a005-480d-a11f-6ad74c3a8cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:22:01.449492 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91492a1-a005-480d-a11f-6ad74c3a8cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:22:01.450462 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91492a1-a005-480d-a11f-6ad74c3a8cc0 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:22:01.451232 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91492a1-a005-480d-a11f-6ad74c3a8cc0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:22:01.451545 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91492a1-a005-480d-a11f-6ad74c3a8cc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:22:01.453000 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91492a1-a005-480d-a11f-6ad74c3a8cc0 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:22:01.459262 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 669/1305] 10.222.0.65 () {56 vars in 1116 bytes} [Thu May 19 21:22:01 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:22:01.466546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711f9e8-3053-46f0-a46f-51ac3723c175 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:22:01.586418 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711f9e8-3053-46f0-a46f-51ac3723c175 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:22:01.634118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd93d72-c493-4672-bbf6-189a45722d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:22:01.634725 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd93d72-c493-4672-bbf6-189a45722d73 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:22:01.635008 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd93d72-c493-4672-bbf6-189a45722d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:22:01.635274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd93d72-c493-4672-bbf6-189a45722d73 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:22:01.636057 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd93d72-c493-4672-bbf6-189a45722d73 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:22:01.637821 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd93d72-c493-4672-bbf6-189a45722d73 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:22:01.657620 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 637/1306] 10.222.0.65 () {60 vars in 1144 bytes} [Thu May 19 21:22:01 2022] POST /identity/v3/endpoints => generated 331 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:22:11.455822 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=30 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:22:11.653315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:23:10.008072 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca1e33-67b1-4fdd-9603-9db3ea6ecc2e None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:10.008958 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca1e33-67b1-4fdd-9603-9db3ea6ecc2e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:10.009382 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca1e33-67b1-4fdd-9603-9db3ea6ecc2e None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:10.015151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 670/1307] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:23:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:23:10.016626 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0093332c-fa1a-4cf0-b8c9-d445b73df8ef None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:10.017087 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0093332c-fa1a-4cf0-b8c9-d445b73df8ef None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:10.017182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0093332c-fa1a-4cf0-b8c9-d445b73df8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:10.100997 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-0093332c-fa1a-4cf0-b8c9-d445b73df8ef None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:23:10.102512 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0093332c-fa1a-4cf0-b8c9-d445b73df8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:23:10.262306 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 638/1308] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:23:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:23:10.321977 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180ab2b-71a0-4fc8-bb08-0e6285065e0a None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:10.322355 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180ab2b-71a0-4fc8-bb08-0e6285065e0a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:10.322623 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180ab2b-71a0-4fc8-bb08-0e6285065e0a None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:10.324210 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 671/1309] 10.222.0.65 () {54 vars in 964 bytes} [Thu May 19 21:23:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:23:10.337665 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d60466-2299-44ce-ac1b-9db546cc4d57 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:10.337951 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d60466-2299-44ce-ac1b-9db546cc4d57 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:10.338817 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d60466-2299-44ce-ac1b-9db546cc4d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:10.451624 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-68d60466-2299-44ce-ac1b-9db546cc4d57 None None] MFA Rules not processed for user `2201e66caa4b4a57be09d845d7cb0023`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:23:10.452985 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-68d60466-2299-44ce-ac1b-9db546cc4d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:23:10.479623 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:23:10.541751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 639/1310] 10.222.0.65 () {58 vars in 1061 bytes} [Thu May 19 21:23:10 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:23:10.550828 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f180ab2b-71a0-4fc8-bb08-0e6285065e0a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:23:10.552900 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f180ab2b-71a0-4fc8-bb08-0e6285065e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:23:10.696262 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f180ab2b-71a0-4fc8-bb08-0e6285065e0a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:23:10.781476 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0ea70-2a3f-48f6-94d7-80f53f518693 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:23:10.782343 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0ea70-2a3f-48f6-94d7-80f53f518693 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:10.782732 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0ea70-2a3f-48f6-94d7-80f53f518693 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:10.783112 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0ea70-2a3f-48f6-94d7-80f53f518693 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:10.784931 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-45d0ea70-2a3f-48f6-94d7-80f53f518693 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:23:10.850271 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:23:10.910364 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0ea70-2a3f-48f6-94d7-80f53f518693 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:23:10.911823 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0ea70-2a3f-48f6-94d7-80f53f518693 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:23:10.933315 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 672/1311] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:23:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:23:13.500799 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2474f178-cdc4-4155-bf64-0b0f4ec4c92d None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:13.501602 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2474f178-cdc4-4155-bf64-0b0f4ec4c92d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:13.501602 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2474f178-cdc4-4155-bf64-0b0f4ec4c92d None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:13.503382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 640/1312] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:23:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:23:13.509613 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5309e69f-618b-4f19-84e1-65aaa31f378f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:13.510198 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5309e69f-618b-4f19-84e1-65aaa31f378f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:13.510198 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5309e69f-618b-4f19-84e1-65aaa31f378f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:13.614588 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-5309e69f-618b-4f19-84e1-65aaa31f378f None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:23:13.615836 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5309e69f-618b-4f19-84e1-65aaa31f378f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:23:13.735298 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 673/1313] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:23:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:23:13.781489 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2474f178-cdc4-4155-bf64-0b0f4ec4c92d None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:23:13.783732 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-2474f178-cdc4-4155-bf64-0b0f4ec4c92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:23:13.929123 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2474f178-cdc4-4155-bf64-0b0f4ec4c92d None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:23:14.016023 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:23:14.019554 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ee718-a7c4-4f6d-8de3-f98b91ffad09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:23:14.020399 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ee718-a7c4-4f6d-8de3-f98b91ffad09 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:14.020887 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ee718-a7c4-4f6d-8de3-f98b91ffad09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:14.021589 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ee718-a7c4-4f6d-8de3-f98b91ffad09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:14.024305 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-cd4ee718-a7c4-4f6d-8de3-f98b91ffad09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:23:14.179821 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4ee718-a7c4-4f6d-8de3-f98b91ffad09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:23:14.181459 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4ee718-a7c4-4f6d-8de3-f98b91ffad09 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:23:14.204470 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 641/1314] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:23:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:23:17.332774 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978a18d-088f-46f1-b5f3-a2c8a0984804 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:17.333516 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978a18d-088f-46f1-b5f3-a2c8a0984804 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:17.333516 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978a18d-088f-46f1-b5f3-a2c8a0984804 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:17.334811 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 674/1315] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:23:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:23:17.340448 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689d4d3-24f3-4ffe-b382-6984ce76a8d4 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:17.341337 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689d4d3-24f3-4ffe-b382-6984ce76a8d4 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:17.341337 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689d4d3-24f3-4ffe-b382-6984ce76a8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:17.413295 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-d689d4d3-24f3-4ffe-b382-6984ce76a8d4 None None] MFA Rules not processed for user `b66dd2c6e7694191810a310a769dc893`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:23:17.414631 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d689d4d3-24f3-4ffe-b382-6984ce76a8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:23:17.475934 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:23:17.534734 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 642/1316] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:23:17 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:23:17.596810 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db7e7b5-bdc1-499a-869e-ce19d44f989e None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:17.597441 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db7e7b5-bdc1-499a-869e-ce19d44f989e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:17.598291 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db7e7b5-bdc1-499a-869e-ce19d44f989e None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:17.601173 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 675/1317] 10.222.0.65 () {54 vars in 964 bytes} [Thu May 19 21:23:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:23:17.611027 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f9af12-37a4-4545-b9c8-936f734b98b7 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:17.611487 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f9af12-37a4-4545-b9c8-936f734b98b7 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:17.611900 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f9af12-37a4-4545-b9c8-936f734b98b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:17.707814 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-e7f9af12-37a4-4545-b9c8-936f734b98b7 None None] MFA Rules not processed for user `2201e66caa4b4a57be09d845d7cb0023`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:23:17.709047 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e7f9af12-37a4-4545-b9c8-936f734b98b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:23:17.811934 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 643/1318] 10.222.0.65 () {58 vars in 1061 bytes} [Thu May 19 21:23:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:23:17.823689 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db7e7b5-bdc1-499a-869e-ce19d44f989e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:23:17.825481 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6db7e7b5-bdc1-499a-869e-ce19d44f989e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:23:17.836242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:23:17.954604 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db7e7b5-bdc1-499a-869e-ce19d44f989e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:23:18.044803 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769390e-5456-4793-a992-3669c27caeb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:23:18.046171 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769390e-5456-4793-a992-3669c27caeb5 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:18.046556 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769390e-5456-4793-a992-3669c27caeb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:18.046977 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769390e-5456-4793-a992-3669c27caeb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:18.049656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b769390e-5456-4793-a992-3669c27caeb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:23:18.138822 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:23:18.197042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b769390e-5456-4793-a992-3669c27caeb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:23:18.199084 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b769390e-5456-4793-a992-3669c27caeb5 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:23:18.221366 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 676/1319] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:23:17 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:23:21.572203 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d9897-ad35-4646-9aec-479e8017c42e None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:21.572847 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d9897-ad35-4646-9aec-479e8017c42e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:21.572847 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d9897-ad35-4646-9aec-479e8017c42e None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:21.574259 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 644/1320] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:23:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:23:21.580572 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ca095-dfb3-465e-a05b-48afe280756e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:21.581095 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ca095-dfb3-465e-a05b-48afe280756e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:21.581190 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ca095-dfb3-465e-a05b-48afe280756e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:21.658106 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f11ca095-dfb3-465e-a05b-48afe280756e None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:23:21.659390 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f11ca095-dfb3-465e-a05b-48afe280756e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:23:21.773299 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 677/1321] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:23:21 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:23:21.819483 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d9897-ad35-4646-9aec-479e8017c42e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:23:21.821178 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-de8d9897-ad35-4646-9aec-479e8017c42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:23:21.890877 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:23:21.954721 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d9897-ad35-4646-9aec-479e8017c42e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:23:22.050494 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ab2e36-ed77-40b2-948e-61c5ee5ef91b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:23:22.051161 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab2e36-ed77-40b2-948e-61c5ee5ef91b service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:22.051445 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab2e36-ed77-40b2-948e-61c5ee5ef91b service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:22.051725 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab2e36-ed77-40b2-948e-61c5ee5ef91b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:22.053424 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-58ab2e36-ed77-40b2-948e-61c5ee5ef91b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:23:22.170508 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:23:22.187318 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ab2e36-ed77-40b2-948e-61c5ee5ef91b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:23:22.188799 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ab2e36-ed77-40b2-948e-61c5ee5ef91b service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:23:22.210601 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 645/1322] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:23:21 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:23:25.570682 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c21df-bc26-40fd-a4fd-bd2d35d6cf43 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:25.571341 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c21df-bc26-40fd-a4fd-bd2d35d6cf43 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:25.572008 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c21df-bc26-40fd-a4fd-bd2d35d6cf43 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:25.573638 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 678/1323] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:23:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:23:25.580652 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f826f-9740-4f88-a299-433d60d5d585 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:25.581310 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f826f-9740-4f88-a299-433d60d5d585 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:25.581517 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f826f-9740-4f88-a299-433d60d5d585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:25.659052 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-483f826f-9740-4f88-a299-433d60d5d585 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:23:25.660370 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-483f826f-9740-4f88-a299-433d60d5d585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:23:25.790309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 646/1324] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:23:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:23:25.827700 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c21df-bc26-40fd-a4fd-bd2d35d6cf43 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:23:25.829645 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c71c21df-bc26-40fd-a4fd-bd2d35d6cf43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:23:25.854833 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:23:25.972625 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c21df-bc26-40fd-a4fd-bd2d35d6cf43 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:23:26.072542 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fbd851-6e27-43f5-8601-422269f2dba3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:23:26.073229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbd851-6e27-43f5-8601-422269f2dba3 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:26.073509 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbd851-6e27-43f5-8601-422269f2dba3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:26.073784 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbd851-6e27-43f5-8601-422269f2dba3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:26.075683 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-08fbd851-6e27-43f5-8601-422269f2dba3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:23:26.166524 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:23:26.222929 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fbd851-6e27-43f5-8601-422269f2dba3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:23:26.224440 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fbd851-6e27-43f5-8601-422269f2dba3 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:23:26.249270 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 679/1325] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:23:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:23:28.546485 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301941c-c7cc-49e6-81ec-58b861220c44 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:28.547039 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301941c-c7cc-49e6-81ec-58b861220c44 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:28.547039 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301941c-c7cc-49e6-81ec-58b861220c44 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:28.548435 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 647/1326] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:23:28.554046 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e7d5e0-5b53-485a-ae05-94d11bb6ad1b None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:28.554513 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e7d5e0-5b53-485a-ae05-94d11bb6ad1b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:28.554767 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e7d5e0-5b53-485a-ae05-94d11bb6ad1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:28.632387 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-28e7d5e0-5b53-485a-ae05-94d11bb6ad1b None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:23:28.633648 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-28e7d5e0-5b53-485a-ae05-94d11bb6ad1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:23:28.741625 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 680/1327] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:23:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:23:28.777467 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301941c-c7cc-49e6-81ec-58b861220c44 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:23:28.779579 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3301941c-c7cc-49e6-81ec-58b861220c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:23:28.844167 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:23:28.898494 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301941c-c7cc-49e6-81ec-58b861220c44 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:23:28.995435 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc35c536-c477-4e48-8cdc-c52468357701 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:23:28.996100 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35c536-c477-4e48-8cdc-c52468357701 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:28.996382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35c536-c477-4e48-8cdc-c52468357701 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:28.996652 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35c536-c477-4e48-8cdc-c52468357701 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:28.998420 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-fc35c536-c477-4e48-8cdc-c52468357701 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:23:29.114211 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:23:29.117154 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc35c536-c477-4e48-8cdc-c52468357701 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:23:29.119032 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc35c536-c477-4e48-8cdc-c52468357701 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:23:29.138616 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 648/1328] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:23:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:23:31.476793 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63110d-feaa-4962-a2b2-dc507eeae5f7 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:31.477490 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63110d-feaa-4962-a2b2-dc507eeae5f7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:31.477490 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63110d-feaa-4962-a2b2-dc507eeae5f7 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:31.478758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 681/1329] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:23:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:23:31.484752 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4a1c7-05e5-4b54-b494-5e32373c60eb None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:31.485245 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4a1c7-05e5-4b54-b494-5e32373c60eb None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:31.485354 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4a1c7-05e5-4b54-b494-5e32373c60eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:31.553206 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-bba4a1c7-05e5-4b54-b494-5e32373c60eb None None] MFA Rules not processed for user `b66dd2c6e7694191810a310a769dc893`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:23:31.554452 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-bba4a1c7-05e5-4b54-b494-5e32373c60eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:23:31.658259 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 649/1330] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:23:31 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:23:31.695771 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63110d-feaa-4962-a2b2-dc507eeae5f7 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:23:31.697333 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2b63110d-feaa-4962-a2b2-dc507eeae5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:23:31.731519 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:23:31.812859 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63110d-feaa-4962-a2b2-dc507eeae5f7 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:23:31.894571 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f5e9c-17df-4057-adb5-9b71763e671b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:23:31.895190 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f5e9c-17df-4057-adb5-9b71763e671b service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:31.895466 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f5e9c-17df-4057-adb5-9b71763e671b service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:31.895763 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f5e9c-17df-4057-adb5-9b71763e671b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:31.897441 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4a9f5e9c-17df-4057-adb5-9b71763e671b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:23:31.980992 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:23:32.016257 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9f5e9c-17df-4057-adb5-9b71763e671b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:23:32.017608 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9f5e9c-17df-4057-adb5-9b71763e671b service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:23:32.037046 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 682/1331] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:23:31 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:23:34.581731 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccea657-31c9-4995-8a35-b2522df10c70 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:34.582299 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccea657-31c9-4995-8a35-b2522df10c70 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:34.582299 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccea657-31c9-4995-8a35-b2522df10c70 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:34.583893 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 650/1332] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:23:34.589332 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222a0d6-4e52-406b-8823-dbef624aa3d8 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:34.589792 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222a0d6-4e52-406b-8823-dbef624aa3d8 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:34.589874 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222a0d6-4e52-406b-8823-dbef624aa3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:34.680216 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d222a0d6-4e52-406b-8823-dbef624aa3d8 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:23:34.681473 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d222a0d6-4e52-406b-8823-dbef624aa3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:23:34.787768 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 683/1333] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:23:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:23:34.823935 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccea657-31c9-4995-8a35-b2522df10c70 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:23:34.825630 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-2ccea657-31c9-4995-8a35-b2522df10c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:23:34.907871 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:23:34.958168 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccea657-31c9-4995-8a35-b2522df10c70 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:23:35.051062 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa2884-0c13-4ab9-a3f5-59def2eb990c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:23:35.051727 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa2884-0c13-4ab9-a3f5-59def2eb990c service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:35.052008 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa2884-0c13-4ab9-a3f5-59def2eb990c service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:35.052281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa2884-0c13-4ab9-a3f5-59def2eb990c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:35.053995 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f7aa2884-0c13-4ab9-a3f5-59def2eb990c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:23:35.182787 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aa2884-0c13-4ab9-a3f5-59def2eb990c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:23:35.184353 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aa2884-0c13-4ab9-a3f5-59def2eb990c service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:23:35.196164 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:23:35.208292 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 651/1334] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:23:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:23:37.541353 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010bba1c-ce98-4a9c-993a-2ae6e3e80f82 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:37.541901 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010bba1c-ce98-4a9c-993a-2ae6e3e80f82 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:37.541901 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010bba1c-ce98-4a9c-993a-2ae6e3e80f82 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:37.543641 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 684/1335] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:23:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:23:37.550260 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3f9d3-1a3e-4b3d-9d71-c3224b212a2d None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:37.550761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3f9d3-1a3e-4b3d-9d71-c3224b212a2d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:37.550852 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3f9d3-1a3e-4b3d-9d71-c3224b212a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:37.635079 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-37f3f9d3-1a3e-4b3d-9d71-c3224b212a2d None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:23:37.636310 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-37f3f9d3-1a3e-4b3d-9d71-c3224b212a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:23:37.745389 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 652/1336] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:23:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:23:37.785759 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010bba1c-ce98-4a9c-993a-2ae6e3e80f82 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:23:37.787481 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-010bba1c-ce98-4a9c-993a-2ae6e3e80f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:23:37.831942 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:23:37.910865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010bba1c-ce98-4a9c-993a-2ae6e3e80f82 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:23:38.000150 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f87ec0-86d2-4985-8645-fe38e7002a6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:23:38.000814 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f87ec0-86d2-4985-8645-fe38e7002a6f service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:38.001096 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f87ec0-86d2-4985-8645-fe38e7002a6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:38.001370 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f87ec0-86d2-4985-8645-fe38e7002a6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:38.003085 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-62f87ec0-86d2-4985-8645-fe38e7002a6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:23:38.089746 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:23:38.117364 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f87ec0-86d2-4985-8645-fe38e7002a6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:23:38.118969 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f87ec0-86d2-4985-8645-fe38e7002a6f service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:23:38.142410 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 685/1337] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:23:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:23:40.911566 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31ad8e-5bb2-4b9c-9a5f-6acf69daca48 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:40.912128 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31ad8e-5bb2-4b9c-9a5f-6acf69daca48 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:40.912128 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31ad8e-5bb2-4b9c-9a5f-6acf69daca48 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:40.913490 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 653/1338] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:23:40.919445 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0817716-ccf1-4440-abf0-ac18d6b0c174 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:40.920165 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0817716-ccf1-4440-abf0-ac18d6b0c174 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:40.920165 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0817716-ccf1-4440-abf0-ac18d6b0c174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:41.001094 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f0817716-ccf1-4440-abf0-ac18d6b0c174 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:23:41.002348 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f0817716-ccf1-4440-abf0-ac18d6b0c174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:23:41.117718 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 686/1339] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:23:40 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:23:41.153566 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be31ad8e-5bb2-4b9c-9a5f-6acf69daca48 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:23:41.155985 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-be31ad8e-5bb2-4b9c-9a5f-6acf69daca48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:23:41.244746 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:23:41.262275 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be31ad8e-5bb2-4b9c-9a5f-6acf69daca48 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:23:41.267118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b80c3-b0ca-45f3-83d7-d0ffb49e2e5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:23:41.267778 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b80c3-b0ca-45f3-83d7-d0ffb49e2e5d service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:41.268039 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b80c3-b0ca-45f3-83d7-d0ffb49e2e5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:41.268324 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b80c3-b0ca-45f3-83d7-d0ffb49e2e5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:41.279022 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b80c3-b0ca-45f3-83d7-d0ffb49e2e5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:23:41.280547 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b80c3-b0ca-45f3-83d7-d0ffb49e2e5d service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:23:41.290648 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 654/1340] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:23:41 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:23:45.829751 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc885f1d-8866-471d-a012-c2c4ad302daa None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:45.830403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc885f1d-8866-471d-a012-c2c4ad302daa None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:45.830403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc885f1d-8866-471d-a012-c2c4ad302daa None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:45.831727 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 687/1341] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:23:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:23:45.837413 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e186e58-86b4-46fe-81b1-c3707ccd123a None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:45.837880 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e186e58-86b4-46fe-81b1-c3707ccd123a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:45.838118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e186e58-86b4-46fe-81b1-c3707ccd123a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:45.869992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-5e186e58-86b4-46fe-81b1-c3707ccd123a None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:23:45.871348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5e186e58-86b4-46fe-81b1-c3707ccd123a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:23:45.953747 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 655/1342] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:23:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:23:45.992813 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc885f1d-8866-471d-a012-c2c4ad302daa None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:23:46.005747 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc885f1d-8866-471d-a012-c2c4ad302daa None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:23:46.009590 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee76c27-aa90-4901-b9fb-3c3caa710165 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:23:46.010232 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee76c27-aa90-4901-b9fb-3c3caa710165 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:46.010571 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee76c27-aa90-4901-b9fb-3c3caa710165 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:46.010849 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee76c27-aa90-4901-b9fb-3c3caa710165 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:46.021153 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee76c27-aa90-4901-b9fb-3c3caa710165 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:23:46.022621 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee76c27-aa90-4901-b9fb-3c3caa710165 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:23:46.033195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 688/1343] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:23:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:23:48.899390 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcdcce4-ab78-44d1-8b94-0959fb33d617 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:48.899981 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcdcce4-ab78-44d1-8b94-0959fb33d617 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:48.899981 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcdcce4-ab78-44d1-8b94-0959fb33d617 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:48.901330 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 656/1344] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:23:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:23:48.907406 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bb80c-c51b-4990-b1a4-eb8b17e14627 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:48.908050 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bb80c-c51b-4990-b1a4-eb8b17e14627 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:48.908050 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bb80c-c51b-4990-b1a4-eb8b17e14627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:48.928292 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e32bb80c-c51b-4990-b1a4-eb8b17e14627 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:23:48.929539 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e32bb80c-c51b-4990-b1a4-eb8b17e14627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:23:48.938795 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 689/1345] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:23:48 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:23:48.976610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcdcce4-ab78-44d1-8b94-0959fb33d617 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:23:48.990176 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcdcce4-ab78-44d1-8b94-0959fb33d617 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:23:48.994574 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4edd0c-5896-4500-9668-2f7c9a31c150 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:23:48.995241 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4edd0c-5896-4500-9668-2f7c9a31c150 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:48.995525 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4edd0c-5896-4500-9668-2f7c9a31c150 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:48.995804 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4edd0c-5896-4500-9668-2f7c9a31c150 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:49.006707 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4edd0c-5896-4500-9668-2f7c9a31c150 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:23:49.008234 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4edd0c-5896-4500-9668-2f7c9a31c150 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:23:49.019999 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 657/1346] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:23:48 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:23:53.361590 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b842a-52e8-4fb8-9f2b-1fe8b51b8a4e None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:53.362195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b842a-52e8-4fb8-9f2b-1fe8b51b8a4e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:53.362195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b842a-52e8-4fb8-9f2b-1fe8b51b8a4e None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:53.363571 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 690/1347] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:23:53.369014 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d6e6b-d2f9-4c43-851b-a58601688334 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:53.369421 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d6e6b-d2f9-4c43-851b-a58601688334 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:53.369566 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d6e6b-d2f9-4c43-851b-a58601688334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:53.389253 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-7b2d6e6b-d2f9-4c43-851b-a58601688334 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:23:53.390538 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7b2d6e6b-d2f9-4c43-851b-a58601688334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:23:53.400034 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 658/1348] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:23:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:23:53.438772 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b842a-52e8-4fb8-9f2b-1fe8b51b8a4e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:23:53.451880 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b842a-52e8-4fb8-9f2b-1fe8b51b8a4e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:23:53.455998 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ffeb5c-9b69-4f71-ac8c-785870fbfee2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:23:53.456671 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ffeb5c-9b69-4f71-ac8c-785870fbfee2 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:53.456949 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ffeb5c-9b69-4f71-ac8c-785870fbfee2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:53.457202 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ffeb5c-9b69-4f71-ac8c-785870fbfee2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:53.468718 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ffeb5c-9b69-4f71-ac8c-785870fbfee2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:23:53.470194 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ffeb5c-9b69-4f71-ac8c-785870fbfee2 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:23:53.481326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 691/1349] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:23:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:23:59.345463 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dabb70-94fc-4902-a1ac-7458e2eccd97 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:59.346397 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dabb70-94fc-4902-a1ac-7458e2eccd97 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:59.346898 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dabb70-94fc-4902-a1ac-7458e2eccd97 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:59.348812 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 659/1350] 10.222.0.65 () {54 vars in 921 bytes} [Thu May 19 21:23:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:23:59.354082 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a2e3b-dd4d-49e9-9ba4-a1af16af0b81 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:23:59.354589 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a2e3b-dd4d-49e9-9ba4-a1af16af0b81 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:23:59.354688 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a2e3b-dd4d-49e9-9ba4-a1af16af0b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:23:59.372645 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:23:59.470174 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d45a2e3b-dd4d-49e9-9ba4-a1af16af0b81 None None] MFA Rules not processed for user `f6143bddc05a46ea972709576c7ad95f`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:23:59.471429 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d45a2e3b-dd4d-49e9-9ba4-a1af16af0b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:23:59.581882 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 692/1351] 10.222.0.65 () {58 vars in 1018 bytes} [Thu May 19 21:23:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:24:03.383107 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=57 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:24:03.941105 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e7b70-d8fc-431d-874a-810e12cca476 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:03.941687 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e7b70-d8fc-431d-874a-810e12cca476 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:03.942051 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e7b70-d8fc-431d-874a-810e12cca476 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:03.943585 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 660/1352] 10.222.0.65 () {54 vars in 921 bytes} [Thu May 19 21:24:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:24:03.949207 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce728ce-7c20-4044-ae7d-c73dcf9a7aba None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:03.949842 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce728ce-7c20-4044-ae7d-c73dcf9a7aba None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:03.950077 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce728ce-7c20-4044-ae7d-c73dcf9a7aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:03.972530 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0ce728ce-7c20-4044-ae7d-c73dcf9a7aba None None] MFA Rules not processed for user `f6143bddc05a46ea972709576c7ad95f`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:24:03.974658 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0ce728ce-7c20-4044-ae7d-c73dcf9a7aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:24:03.984976 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 693/1353] 10.222.0.65 () {58 vars in 1018 bytes} [Thu May 19 21:24:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:24:05.562105 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d341145-0623-46eb-932e-d3ab5dc5ce8c None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:05.562854 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d341145-0623-46eb-932e-d3ab5dc5ce8c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:05.562854 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d341145-0623-46eb-932e-d3ab5dc5ce8c None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:05.564238 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 661/1354] 10.222.0.65 () {54 vars in 921 bytes} [Thu May 19 21:24:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:24:05.569941 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5be573-8665-4bcd-8f59-510579730a56 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:05.570471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5be573-8665-4bcd-8f59-510579730a56 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:05.570471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5be573-8665-4bcd-8f59-510579730a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:05.589227 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-be5be573-8665-4bcd-8f59-510579730a56 None None] MFA Rules not processed for user `f6143bddc05a46ea972709576c7ad95f`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:24:05.590450 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-be5be573-8665-4bcd-8f59-510579730a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:24:05.599730 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 694/1355] 10.222.0.65 () {58 vars in 1018 bytes} [Thu May 19 21:24:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:24:09.545156 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789d4bb-a06b-485b-a635-ffd1e6849c28 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:09.545779 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789d4bb-a06b-485b-a635-ffd1e6849c28 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:09.545883 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789d4bb-a06b-485b-a635-ffd1e6849c28 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:09.556229 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 662/1356] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:24:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:24:09.559300 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9e6ac-746f-4c79-a4b2-9f8bb4c60258 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:09.560562 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9e6ac-746f-4c79-a4b2-9f8bb4c60258 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:09.560991 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9e6ac-746f-4c79-a4b2-9f8bb4c60258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:09.596148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-edf9e6ac-746f-4c79-a4b2-9f8bb4c60258 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:24:09.598042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-edf9e6ac-746f-4c79-a4b2-9f8bb4c60258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:24:09.614107 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 695/1357] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:24:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:24:09.677887 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56bf95-2f05-497c-9dc1-735a40649c59 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:09.678232 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56bf95-2f05-497c-9dc1-735a40649c59 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:09.678540 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56bf95-2f05-497c-9dc1-735a40649c59 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:09.679954 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 663/1358] 10.222.0.65 () {54 vars in 961 bytes} [Thu May 19 21:24:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:24:09.694886 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e28aec-ea43-44d0-a8c9-9eb31e37bf00 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:09.695446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e28aec-ea43-44d0-a8c9-9eb31e37bf00 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:09.695878 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e28aec-ea43-44d0-a8c9-9eb31e37bf00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:09.750341 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f0e28aec-ea43-44d0-a8c9-9eb31e37bf00 None None] MFA Rules not processed for user `e254671b21fe4661a381b9521810cf80`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:24:09.751820 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f0e28aec-ea43-44d0-a8c9-9eb31e37bf00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:24:09.815316 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:24:09.837983 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 696/1359] 10.222.0.65 () {58 vars in 1058 bytes} [Thu May 19 21:24:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:24:09.847688 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb56bf95-2f05-497c-9dc1-735a40649c59 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:24:09.864455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb56bf95-2f05-497c-9dc1-735a40649c59 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:24:09.869788 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40625a83-b494-4e6a-bd70-7aab55e673f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:24:09.870470 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40625a83-b494-4e6a-bd70-7aab55e673f3 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:09.870778 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40625a83-b494-4e6a-bd70-7aab55e673f3 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:09.871052 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40625a83-b494-4e6a-bd70-7aab55e673f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:09.881397 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40625a83-b494-4e6a-bd70-7aab55e673f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:24:09.883210 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40625a83-b494-4e6a-bd70-7aab55e673f3 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:24:09.894315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 664/1360] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:24:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:24:12.547550 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d65ad-e0c5-47d1-af61-899dbba0649e None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:12.548305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d65ad-e0c5-47d1-af61-899dbba0649e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:12.548305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d65ad-e0c5-47d1-af61-899dbba0649e None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:12.549418 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 697/1361] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:24:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:24:12.558475 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce346a3-a67a-4cb5-8e48-31efbfb12682 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:12.559669 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce346a3-a67a-4cb5-8e48-31efbfb12682 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:12.560068 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce346a3-a67a-4cb5-8e48-31efbfb12682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:12.585639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-cce346a3-a67a-4cb5-8e48-31efbfb12682 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:24:12.587729 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-cce346a3-a67a-4cb5-8e48-31efbfb12682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:24:12.598315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 665/1362] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:24:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:24:12.649925 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3d65ad-e0c5-47d1-af61-899dbba0649e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:24:12.665430 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3d65ad-e0c5-47d1-af61-899dbba0649e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:24:12.671779 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085fc24d-1655-4c95-88c7-d81c1eb58b89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:24:12.672465 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fc24d-1655-4c95-88c7-d81c1eb58b89 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:12.672744 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fc24d-1655-4c95-88c7-d81c1eb58b89 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:12.673023 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fc24d-1655-4c95-88c7-d81c1eb58b89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:12.684513 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085fc24d-1655-4c95-88c7-d81c1eb58b89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:24:12.687265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085fc24d-1655-4c95-88c7-d81c1eb58b89 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:24:12.699754 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 698/1363] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:24:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:24:15.578168 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b968d-7f9a-4e4d-a9ef-469944293513 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:15.579125 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b968d-7f9a-4e4d-a9ef-469944293513 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:15.579125 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b968d-7f9a-4e4d-a9ef-469944293513 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:15.580560 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 666/1364] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:24:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:24:15.586895 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba30576-f92f-4feb-b8dd-90d4e79bf283 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:15.587670 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba30576-f92f-4feb-b8dd-90d4e79bf283 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:15.587670 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba30576-f92f-4feb-b8dd-90d4e79bf283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:15.609419 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0ba30576-f92f-4feb-b8dd-90d4e79bf283 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:24:15.610909 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0ba30576-f92f-4feb-b8dd-90d4e79bf283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:24:15.619998 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 699/1365] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:24:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:24:15.675651 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36c2cb-8d6f-4253-b417-df0f72ae6add None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:15.676125 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36c2cb-8d6f-4253-b417-df0f72ae6add None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:15.676825 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36c2cb-8d6f-4253-b417-df0f72ae6add None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:15.681893 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 667/1366] 10.222.0.65 () {54 vars in 961 bytes} [Thu May 19 21:24:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:24:15.691680 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ca3fc-2211-4644-8494-485e53befe24 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:15.692196 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ca3fc-2211-4644-8494-485e53befe24 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:15.692702 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ca3fc-2211-4644-8494-485e53befe24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:15.723967 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9a4ca3fc-2211-4644-8494-485e53befe24 None None] MFA Rules not processed for user `e254671b21fe4661a381b9521810cf80`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:24:15.725861 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9a4ca3fc-2211-4644-8494-485e53befe24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:24:15.741191 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 700/1367] 10.222.0.65 () {58 vars in 1058 bytes} [Thu May 19 21:24:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:24:15.750806 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36c2cb-8d6f-4253-b417-df0f72ae6add None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:24:15.776626 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36c2cb-8d6f-4253-b417-df0f72ae6add None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:24:15.787889 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d138c08a-06b4-448a-a6b3-06451a350696 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:24:15.788718 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d138c08a-06b4-448a-a6b3-06451a350696 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:15.788859 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d138c08a-06b4-448a-a6b3-06451a350696 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:15.789106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d138c08a-06b4-448a-a6b3-06451a350696 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:15.800480 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d138c08a-06b4-448a-a6b3-06451a350696 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:24:15.802122 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d138c08a-06b4-448a-a6b3-06451a350696 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:24:15.824654 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 668/1368] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:24:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:24:18.406085 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97285756-9bd6-4cd6-b12e-f2be10bd981f None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:18.406979 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97285756-9bd6-4cd6-b12e-f2be10bd981f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:18.407456 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97285756-9bd6-4cd6-b12e-f2be10bd981f None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:18.410198 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 701/1369] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:24:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:24:18.420273 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643e123-b418-431f-a59c-568c5e9cdb1f None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:18.420841 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643e123-b418-431f-a59c-568c5e9cdb1f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:18.421226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643e123-b418-431f-a59c-568c5e9cdb1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:18.452073 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b643e123-b418-431f-a59c-568c5e9cdb1f None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:24:18.454206 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b643e123-b418-431f-a59c-568c5e9cdb1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:24:18.465273 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 669/1370] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:24:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:24:18.540732 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97285756-9bd6-4cd6-b12e-f2be10bd981f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:24:18.558480 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97285756-9bd6-4cd6-b12e-f2be10bd981f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:24:18.568501 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a141dfd3-0c01-4331-9b43-43ceb79af203 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:24:18.569685 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141dfd3-0c01-4331-9b43-43ceb79af203 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:18.570164 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141dfd3-0c01-4331-9b43-43ceb79af203 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:18.570696 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141dfd3-0c01-4331-9b43-43ceb79af203 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:18.587754 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a141dfd3-0c01-4331-9b43-43ceb79af203 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:24:18.590959 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a141dfd3-0c01-4331-9b43-43ceb79af203 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:24:18.611208 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 702/1371] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:24:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:24:19.181778 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126a915-2224-4791-aa19-eb4cec70a2aa None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:19.182212 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126a915-2224-4791-aa19-eb4cec70a2aa None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:19.182376 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126a915-2224-4791-aa19-eb4cec70a2aa None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:19.184061 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 670/1372] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:24:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:24:19.190422 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15939c4a-f7b8-431e-9ea2-43247b8a570b None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:19.191007 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15939c4a-f7b8-431e-9ea2-43247b8a570b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:19.191489 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15939c4a-f7b8-431e-9ea2-43247b8a570b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:19.211432 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-15939c4a-f7b8-431e-9ea2-43247b8a570b None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:24:19.213005 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-15939c4a-f7b8-431e-9ea2-43247b8a570b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:24:19.221761 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 703/1373] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:24:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:24:19.506370 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ac9fd-13da-4ead-bac7-9e30951f4f0b None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:19.507160 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ac9fd-13da-4ead-bac7-9e30951f4f0b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:19.507160 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ac9fd-13da-4ead-bac7-9e30951f4f0b None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:19.509035 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 671/1374] 10.222.0.65 () {54 vars in 963 bytes} [Thu May 19 21:24:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:24:19.516611 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6c482-631d-46ce-bca3-1c151987e077 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:19.517477 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6c482-631d-46ce-bca3-1c151987e077 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:19.517651 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6c482-631d-46ce-bca3-1c151987e077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:19.589624 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-70d6c482-631d-46ce-bca3-1c151987e077 None None] MFA Rules not processed for user `51852131d4e5448f8152a51159bd78e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:24:19.591344 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-70d6c482-631d-46ce-bca3-1c151987e077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:24:19.699712 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 704/1375] 10.222.0.65 () {58 vars in 1060 bytes} [Thu May 19 21:24:19 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:24:19.710800 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336ac9fd-13da-4ead-bac7-9e30951f4f0b None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:24:19.727902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336ac9fd-13da-4ead-bac7-9e30951f4f0b None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:24:19.734579 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abec607-cf19-4f2a-94d4-b20305290c04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:24:19.735754 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abec607-cf19-4f2a-94d4-b20305290c04 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:19.736280 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abec607-cf19-4f2a-94d4-b20305290c04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:19.736810 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abec607-cf19-4f2a-94d4-b20305290c04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:19.752821 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abec607-cf19-4f2a-94d4-b20305290c04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:24:19.755731 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abec607-cf19-4f2a-94d4-b20305290c04 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:24:19.775062 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 672/1376] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:24:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:24:21.705048 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61d7ef-fada-4314-8b6e-595683fabe9e None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:21.705626 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61d7ef-fada-4314-8b6e-595683fabe9e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:21.706008 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61d7ef-fada-4314-8b6e-595683fabe9e None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:21.713710 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 705/1377] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:24:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:24:21.717049 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3fce3-8115-4c61-a571-fd070eeb1ecd None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:21.717848 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3fce3-8115-4c61-a571-fd070eeb1ecd None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:21.718510 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3fce3-8115-4c61-a571-fd070eeb1ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:21.746843 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b2c3fce3-8115-4c61-a571-fd070eeb1ecd None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:24:21.749031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b2c3fce3-8115-4c61-a571-fd070eeb1ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:24:21.762872 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 673/1378] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:24:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:24:21.805568 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d61d7ef-fada-4314-8b6e-595683fabe9e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:24:21.823616 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d61d7ef-fada-4314-8b6e-595683fabe9e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:24:21.829602 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20523ef7-ac12-443a-adf6-1e03d519e415 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:24:21.830594 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20523ef7-ac12-443a-adf6-1e03d519e415 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:21.831046 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20523ef7-ac12-443a-adf6-1e03d519e415 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:21.831325 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20523ef7-ac12-443a-adf6-1e03d519e415 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:21.841705 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20523ef7-ac12-443a-adf6-1e03d519e415 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:24:21.843888 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20523ef7-ac12-443a-adf6-1e03d519e415 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:24:21.862314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 706/1379] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:24:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:24:24.622045 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339e58e-e9f4-486a-8f6f-ebca7af4c152 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:24.622873 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339e58e-e9f4-486a-8f6f-ebca7af4c152 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:24.623131 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339e58e-e9f4-486a-8f6f-ebca7af4c152 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:24.624731 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 674/1380] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:24:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:24:24.631892 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f9e2e8-05aa-4b29-b7b0-61469c4972db None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:24.632413 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f9e2e8-05aa-4b29-b7b0-61469c4972db None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:24.632494 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f9e2e8-05aa-4b29-b7b0-61469c4972db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:24.659155 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-51f9e2e8-05aa-4b29-b7b0-61469c4972db None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:24:24.660379 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-51f9e2e8-05aa-4b29-b7b0-61469c4972db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:24:24.670691 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 707/1381] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:24:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:24:24.733057 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8339e58e-e9f4-486a-8f6f-ebca7af4c152 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:24:24.752786 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8339e58e-e9f4-486a-8f6f-ebca7af4c152 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:24:24.759584 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7f21e4-4dd7-458f-a88e-d4df38bf3a11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:24:24.760418 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f21e4-4dd7-458f-a88e-d4df38bf3a11 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:24.760806 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f21e4-4dd7-458f-a88e-d4df38bf3a11 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:24.761180 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f21e4-4dd7-458f-a88e-d4df38bf3a11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:24.778222 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7f21e4-4dd7-458f-a88e-d4df38bf3a11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:24:24.781471 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7f21e4-4dd7-458f-a88e-d4df38bf3a11 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:24:24.799868 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 675/1382] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:24:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:24:27.181798 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732fd4c-933b-4f2f-8648-365915cc6cb2 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:27.182361 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732fd4c-933b-4f2f-8648-365915cc6cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:27.182361 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732fd4c-933b-4f2f-8648-365915cc6cb2 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:27.183858 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 708/1383] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:24:27.190370 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0dd053-6708-4044-b056-797580dd8e63 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:27.190860 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0dd053-6708-4044-b056-797580dd8e63 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:27.190940 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0dd053-6708-4044-b056-797580dd8e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:27.211158 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-3a0dd053-6708-4044-b056-797580dd8e63 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:24:27.212471 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3a0dd053-6708-4044-b056-797580dd8e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:24:27.222076 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 676/1384] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:24:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:24:27.263944 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732fd4c-933b-4f2f-8648-365915cc6cb2 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:24:27.278315 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732fd4c-933b-4f2f-8648-365915cc6cb2 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:24:27.284856 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ab644-e449-4b30-9455-49f42e2bc678 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:24:27.285533 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ab644-e449-4b30-9455-49f42e2bc678 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:27.285784 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ab644-e449-4b30-9455-49f42e2bc678 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:27.286287 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ab644-e449-4b30-9455-49f42e2bc678 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:27.296441 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6ab644-e449-4b30-9455-49f42e2bc678 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:24:27.298037 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6ab644-e449-4b30-9455-49f42e2bc678 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:24:27.308800 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 709/1385] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:24:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:24:29.636412 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16067d9d-90f4-4942-b7e9-5d0b06646094 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:29.637103 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16067d9d-90f4-4942-b7e9-5d0b06646094 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:29.637103 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16067d9d-90f4-4942-b7e9-5d0b06646094 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:29.639076 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 677/1386] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:24:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:24:29.645419 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ecf187-ce20-4d43-8fe1-337575e55b0d None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:29.646056 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ecf187-ce20-4d43-8fe1-337575e55b0d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:29.646056 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ecf187-ce20-4d43-8fe1-337575e55b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:29.667294 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c9ecf187-ce20-4d43-8fe1-337575e55b0d None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:24:29.668654 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c9ecf187-ce20-4d43-8fe1-337575e55b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:24:29.678297 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 710/1387] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:24:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:24:29.728578 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16067d9d-90f4-4942-b7e9-5d0b06646094 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:24:29.748594 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16067d9d-90f4-4942-b7e9-5d0b06646094 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:24:29.755267 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11189553-ef27-4eb3-825f-54af71e0ebd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:24:29.756036 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11189553-ef27-4eb3-825f-54af71e0ebd2 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:29.756337 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11189553-ef27-4eb3-825f-54af71e0ebd2 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:29.756630 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11189553-ef27-4eb3-825f-54af71e0ebd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:29.767673 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11189553-ef27-4eb3-825f-54af71e0ebd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:24:29.769913 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11189553-ef27-4eb3-825f-54af71e0ebd2 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:24:29.783547 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 678/1388] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:24:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:24:32.267133 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d70e6-d631-45be-b553-c43014fc929e None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:32.267735 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d70e6-d631-45be-b553-c43014fc929e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:32.267735 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d70e6-d631-45be-b553-c43014fc929e None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:32.269240 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 711/1389] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:24:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:24:32.275596 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4883de-8297-451b-b3e8-9398b93f42c2 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:32.276125 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4883de-8297-451b-b3e8-9398b93f42c2 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:32.276125 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4883de-8297-451b-b3e8-9398b93f42c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:32.302660 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-ba4883de-8297-451b-b3e8-9398b93f42c2 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:24:32.304628 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ba4883de-8297-451b-b3e8-9398b93f42c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:24:32.318913 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 679/1390] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:24:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:24:32.374599 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d70e6-d631-45be-b553-c43014fc929e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:24:32.392298 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d70e6-d631-45be-b553-c43014fc929e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:24:32.398510 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84513103-7359-4504-b194-9dbf3098c4e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:24:32.399228 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84513103-7359-4504-b194-9dbf3098c4e4 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:32.399520 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84513103-7359-4504-b194-9dbf3098c4e4 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:32.399848 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84513103-7359-4504-b194-9dbf3098c4e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:32.412329 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84513103-7359-4504-b194-9dbf3098c4e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:24:32.414070 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84513103-7359-4504-b194-9dbf3098c4e4 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:24:32.425637 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 712/1391] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:24:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:24:34.989192 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd327c6-4c92-4055-8c4e-d1df5cdecf48 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:34.989817 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd327c6-4c92-4055-8c4e-d1df5cdecf48 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:34.989817 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd327c6-4c92-4055-8c4e-d1df5cdecf48 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:34.991349 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 680/1392] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:24:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:24:34.997652 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e29a6-3a0c-4944-8ef7-c053a916ed76 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:34.998522 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e29a6-3a0c-4944-8ef7-c053a916ed76 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:34.998522 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e29a6-3a0c-4944-8ef7-c053a916ed76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:35.021017 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b17e29a6-3a0c-4944-8ef7-c053a916ed76 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:24:35.022682 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b17e29a6-3a0c-4944-8ef7-c053a916ed76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:24:35.035239 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 713/1393] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:24:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:24:35.081531 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd327c6-4c92-4055-8c4e-d1df5cdecf48 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:24:35.091320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=46 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:24:35.101138 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd327c6-4c92-4055-8c4e-d1df5cdecf48 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:24:35.108410 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e32e777-b356-4f1b-826a-55dd85884d0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:24:35.109236 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32e777-b356-4f1b-826a-55dd85884d0e service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:35.109546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32e777-b356-4f1b-826a-55dd85884d0e service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:35.109951 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32e777-b356-4f1b-826a-55dd85884d0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:35.120198 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e32e777-b356-4f1b-826a-55dd85884d0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:24:35.121795 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e32e777-b356-4f1b-826a-55dd85884d0e service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:24:35.133363 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 681/1394] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:24:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:24:37.051402 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96645a3-9fdb-42c1-9d74-58e686522fc6 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:37.052786 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96645a3-9fdb-42c1-9d74-58e686522fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:37.053472 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96645a3-9fdb-42c1-9d74-58e686522fc6 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:37.056390 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 714/1395] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:24:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:24:37.066516 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5203cd2-f9d4-4410-bb34-5e4f4242cca4 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:37.067165 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5203cd2-f9d4-4410-bb34-5e4f4242cca4 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:37.067413 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5203cd2-f9d4-4410-bb34-5e4f4242cca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:37.096761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-d5203cd2-f9d4-4410-bb34-5e4f4242cca4 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:24:37.098945 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d5203cd2-f9d4-4410-bb34-5e4f4242cca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:24:37.113620 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 682/1396] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:24:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:24:37.217582 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd48a6-3001-49ec-bca9-7db0a20f6366 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:37.217846 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd48a6-3001-49ec-bca9-7db0a20f6366 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:37.218137 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd48a6-3001-49ec-bca9-7db0a20f6366 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:37.219608 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 715/1397] 10.222.0.65 () {54 vars in 962 bytes} [Thu May 19 21:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:24:37.225400 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb01e2-80c4-470e-8f30-405bfa942540 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:37.225693 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb01e2-80c4-470e-8f30-405bfa942540 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:37.225931 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb01e2-80c4-470e-8f30-405bfa942540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:37.287677 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-bbfb01e2-80c4-470e-8f30-405bfa942540 None None] MFA Rules not processed for user `ea874ca4088d4e6c83c9d0497323cf6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:24:37.289021 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-bbfb01e2-80c4-470e-8f30-405bfa942540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:24:37.363901 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 683/1398] 10.222.0.65 () {58 vars in 1059 bytes} [Thu May 19 21:24:37 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:24:37.372012 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acd48a6-3001-49ec-bca9-7db0a20f6366 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:24:37.386548 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acd48a6-3001-49ec-bca9-7db0a20f6366 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:24:37.391324 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac903da-4398-465d-8e70-d8a6e37b8dcc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea874ca4088d4e6c83c9d0497323cf6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:24:37.392005 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac903da-4398-465d-8e70-d8a6e37b8dcc service glance] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:37.392285 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac903da-4398-465d-8e70-d8a6e37b8dcc service glance] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:37.392561 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac903da-4398-465d-8e70-d8a6e37b8dcc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:37.403198 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac903da-4398-465d-8e70-d8a6e37b8dcc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:24:37.405412 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac903da-4398-465d-8e70-d8a6e37b8dcc service glance] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:24:37.416041 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 716/1399] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:24:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:24:37.495285 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553d2bd-80f1-4b66-98bb-beef8f65e70d None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:37.495637 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553d2bd-80f1-4b66-98bb-beef8f65e70d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:37.495778 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553d2bd-80f1-4b66-98bb-beef8f65e70d None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:37.497216 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 684/1400] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:24:37.503230 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df866074-d73b-4304-ae40-dcca1a8f0b89 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:37.503520 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df866074-d73b-4304-ae40-dcca1a8f0b89 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:37.503816 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df866074-d73b-4304-ae40-dcca1a8f0b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:37.516468 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:24:37.523135 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-df866074-d73b-4304-ae40-dcca1a8f0b89 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:24:37.524465 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-df866074-d73b-4304-ae40-dcca1a8f0b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:24:37.532902 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 717/1401] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:24:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:24:37.577526 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553d2bd-80f1-4b66-98bb-beef8f65e70d None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:24:37.592606 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553d2bd-80f1-4b66-98bb-beef8f65e70d None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:24:37.598869 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec0c13-034d-485d-b571-59636323df8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:24:37.599607 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec0c13-034d-485d-b571-59636323df8a service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:37.599912 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec0c13-034d-485d-b571-59636323df8a service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:37.600188 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec0c13-034d-485d-b571-59636323df8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:37.612103 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ec0c13-034d-485d-b571-59636323df8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:24:37.614014 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ec0c13-034d-485d-b571-59636323df8a service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:24:37.627278 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 685/1402] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:24:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:24:37.990277 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb9a01-2055-4d87-95d4-b80cb62b85be None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:37.990473 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb9a01-2055-4d87-95d4-b80cb62b85be None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:37.990745 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb9a01-2055-4d87-95d4-b80cb62b85be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:38.045594 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a5fb9a01-2055-4d87-95d4-b80cb62b85be None None] MFA Rules not processed for user `eba91dff7cac401e81f6fb6391874af1`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:24:38.046863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a5fb9a01-2055-4d87-95d4-b80cb62b85be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:24:38.120652 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 718/1403] 10.222.0.65 () {58 vars in 1009 bytes} [Thu May 19 21:24:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:24:38.163329 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b84507-a0a0-43e7-b941-ea4682ec18d6 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:38.163874 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b84507-a0a0-43e7-b941-ea4682ec18d6 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:38.163874 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b84507-a0a0-43e7-b941-ea4682ec18d6 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:38.165253 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 686/1404] 10.222.0.65 () {54 vars in 942 bytes} [Thu May 19 21:24:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:24:38.172494 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c579d7-69c2-4427-bfbb-0690990df7c8 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:38.172801 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c579d7-69c2-4427-bfbb-0690990df7c8 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:38.173081 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c579d7-69c2-4427-bfbb-0690990df7c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:38.224771 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-21c579d7-69c2-4427-bfbb-0690990df7c8 None None] MFA Rules not processed for user `7e07984b36934aff9052541ef2a56f36`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:24:38.225993 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-21c579d7-69c2-4427-bfbb-0690990df7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:24:38.301511 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 719/1405] 10.222.0.65 () {58 vars in 1039 bytes} [Thu May 19 21:24:38 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:24:38.311456 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b84507-a0a0-43e7-b941-ea4682ec18d6 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:24:38.327478 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b84507-a0a0-43e7-b941-ea4682ec18d6 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:24:38.332753 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2efa7c-4bf9-46a2-a3a3-e222655cd5f6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e07984b36934aff9052541ef2a56f36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:24:38.333396 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2efa7c-4bf9-46a2-a3a3-e222655cd5f6 service swift] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:38.333680 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2efa7c-4bf9-46a2-a3a3-e222655cd5f6 service swift] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:38.333959 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2efa7c-4bf9-46a2-a3a3-e222655cd5f6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:38.343034 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2efa7c-4bf9-46a2-a3a3-e222655cd5f6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:24:38.344676 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2efa7c-4bf9-46a2-a3a3-e222655cd5f6 service swift] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:24:38.353389 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 687/1406] 10.222.0.65 () {60 vars in 1384 bytes} [Thu May 19 21:24:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 21:24:39.983170 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e2f6f-86b3-4ceb-a76d-f772d6514be2 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:39.983721 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e2f6f-86b3-4ceb-a76d-f772d6514be2 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:39.983721 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e2f6f-86b3-4ceb-a76d-f772d6514be2 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:39.985170 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 720/1407] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:24:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:24:39.990923 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803b685-af7d-4f0e-a80c-82db222a2dd1 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:39.991447 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803b685-af7d-4f0e-a80c-82db222a2dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:39.991447 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803b685-af7d-4f0e-a80c-82db222a2dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:40.013049 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-4803b685-af7d-4f0e-a80c-82db222a2dd1 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:24:40.014498 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4803b685-af7d-4f0e-a80c-82db222a2dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:24:40.025805 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 688/1408] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:24:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:24:40.073973 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6e2f6f-86b3-4ceb-a76d-f772d6514be2 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:24:40.092556 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6e2f6f-86b3-4ceb-a76d-f772d6514be2 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:24:40.099768 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c84d1-7557-4c68-82c7-1c754a53e4e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:24:40.100798 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c84d1-7557-4c68-82c7-1c754a53e4e9 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:24:40.101264 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c84d1-7557-4c68-82c7-1c754a53e4e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:24:40.101537 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c84d1-7557-4c68-82c7-1c754a53e4e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:24:40.112456 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6c84d1-7557-4c68-82c7-1c754a53e4e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:24:40.114189 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6c84d1-7557-4c68-82c7-1c754a53e4e9 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:24:40.126462 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 721/1409] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:24:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:24:50.006448 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=61 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:24:50.120599 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=90 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:25:02.073692 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8411d0-fe56-434f-aa7d-a92a9e261ccb None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:02.074435 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8411d0-fe56-434f-aa7d-a92a9e261ccb None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:02.074435 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8411d0-fe56-434f-aa7d-a92a9e261ccb None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:02.075776 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 689/1410] 10.222.0.65 () {54 vars in 918 bytes} [Thu May 19 21:25:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:25:02.081626 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84130d5c-4992-4ab7-975d-058d1495cdc6 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:02.082379 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84130d5c-4992-4ab7-975d-058d1495cdc6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:02.082379 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84130d5c-4992-4ab7-975d-058d1495cdc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:02.101520 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-84130d5c-4992-4ab7-975d-058d1495cdc6 None None] MFA Rules not processed for user `f6143bddc05a46ea972709576c7ad95f`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:25:02.102697 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-84130d5c-4992-4ab7-975d-058d1495cdc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:25:02.112314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 722/1411] 10.222.0.65 () {58 vars in 1015 bytes} [Thu May 19 21:25:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:25:02.127930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85158e23-1e04-4d42-b1d8-fe1b557bcb82 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:02.128217 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85158e23-1e04-4d42-b1d8-fe1b557bcb82 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:02.128495 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85158e23-1e04-4d42-b1d8-fe1b557bcb82 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:02.129891 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 690/1412] 10.222.0.65 () {54 vars in 918 bytes} [Thu May 19 21:25:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:25:02.134919 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f86a56-3364-43fb-b68e-98e549fc0a55 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:02.135201 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f86a56-3364-43fb-b68e-98e549fc0a55 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:02.135475 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f86a56-3364-43fb-b68e-98e549fc0a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:02.154647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c2f86a56-3364-43fb-b68e-98e549fc0a55 None None] MFA Rules not processed for user `e254671b21fe4661a381b9521810cf80`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:25:02.155902 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c2f86a56-3364-43fb-b68e-98e549fc0a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:25:02.165323 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 723/1413] 10.222.0.65 () {58 vars in 1015 bytes} [Thu May 19 21:25:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:25:04.997442 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf80e0-2267-43f3-8371-85381828f041 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:04.998376 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf80e0-2267-43f3-8371-85381828f041 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:04.998376 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf80e0-2267-43f3-8371-85381828f041 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:05.000049 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 691/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 21:25:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:25:05.006124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d619ddf3-223a-4395-9069-91eed52b047e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:05.006738 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d619ddf3-223a-4395-9069-91eed52b047e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:05.006738 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d619ddf3-223a-4395-9069-91eed52b047e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:05.026592 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d619ddf3-223a-4395-9069-91eed52b047e None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:25:05.027826 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d619ddf3-223a-4395-9069-91eed52b047e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:25:05.036733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 724/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 21:25:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:25:05.084271 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf80e0-2267-43f3-8371-85381828f041 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:25:05.099745 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf80e0-2267-43f3-8371-85381828f041 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:25:05.105258 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeae139-2149-41b3-b8a5-de88cc80e355 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:25:05.105923 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeae139-2149-41b3-b8a5-de88cc80e355 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:05.106191 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeae139-2149-41b3-b8a5-de88cc80e355 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:05.106556 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeae139-2149-41b3-b8a5-de88cc80e355 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:05.120625 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeae139-2149-41b3-b8a5-de88cc80e355 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:25:05.123241 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeae139-2149-41b3-b8a5-de88cc80e355 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:25:05.134153 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 692/1416] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:25:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:25:06.183180 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5875ad7-57d1-4f9e-ba9f-856db4a04d24 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:06.183734 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5875ad7-57d1-4f9e-ba9f-856db4a04d24 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:06.183734 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5875ad7-57d1-4f9e-ba9f-856db4a04d24 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:06.185862 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 725/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 21:25:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:25:06.191619 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9077e70-f88f-461d-bc68-e09473cb86ec None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:06.192281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9077e70-f88f-461d-bc68-e09473cb86ec None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:06.192281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9077e70-f88f-461d-bc68-e09473cb86ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:06.213703 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-d9077e70-f88f-461d-bc68-e09473cb86ec None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:25:06.214964 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d9077e70-f88f-461d-bc68-e09473cb86ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:25:06.223743 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 693/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 21:25:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:25:06.379081 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5875ad7-57d1-4f9e-ba9f-856db4a04d24 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:25:06.396660 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5875ad7-57d1-4f9e-ba9f-856db4a04d24 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:25:06.402617 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ecf572-4642-48df-beb1-5d8298484b16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:25:06.403464 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ecf572-4642-48df-beb1-5d8298484b16 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:06.404413 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ecf572-4642-48df-beb1-5d8298484b16 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:06.404705 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ecf572-4642-48df-beb1-5d8298484b16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:06.413948 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ecf572-4642-48df-beb1-5d8298484b16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:25:06.415628 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ecf572-4642-48df-beb1-5d8298484b16 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:25:06.427269 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 726/1419] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:25:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:25:07.407206 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c78499-1a16-4f30-b24b-f20a19971c9d None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:07.407931 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c78499-1a16-4f30-b24b-f20a19971c9d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:07.407931 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c78499-1a16-4f30-b24b-f20a19971c9d None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:07.409464 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 694/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 21:25:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:25:07.414795 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf80f05-b1ea-426b-ae2f-b9f04ac6c90e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:07.415353 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf80f05-b1ea-426b-ae2f-b9f04ac6c90e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:07.415353 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf80f05-b1ea-426b-ae2f-b9f04ac6c90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:07.437395 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-4cf80f05-b1ea-426b-ae2f-b9f04ac6c90e None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:25:07.439423 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4cf80f05-b1ea-426b-ae2f-b9f04ac6c90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:25:07.452684 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 727/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 21:25:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:25:07.618408 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c78499-1a16-4f30-b24b-f20a19971c9d None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:25:07.634383 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c78499-1a16-4f30-b24b-f20a19971c9d None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:25:07.641181 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fa2b13-8de9-4ebd-b61f-9ffbb8cb9b57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:25:07.642060 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa2b13-8de9-4ebd-b61f-9ffbb8cb9b57 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:07.642475 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa2b13-8de9-4ebd-b61f-9ffbb8cb9b57 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:07.642849 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa2b13-8de9-4ebd-b61f-9ffbb8cb9b57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:07.652277 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fa2b13-8de9-4ebd-b61f-9ffbb8cb9b57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:25:07.654216 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fa2b13-8de9-4ebd-b61f-9ffbb8cb9b57 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:25:07.665048 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 695/1422] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:25:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:25:08.688565 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643fad7-13ef-4384-8da1-abb9de625729 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:08.689123 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643fad7-13ef-4384-8da1-abb9de625729 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:08.689123 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643fad7-13ef-4384-8da1-abb9de625729 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:08.690465 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 728/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 21:25:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:25:08.696038 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d9acd-0768-4c09-b7de-20e90f957c04 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:08.696576 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d9acd-0768-4c09-b7de-20e90f957c04 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:08.696576 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d9acd-0768-4c09-b7de-20e90f957c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:08.718210 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-850d9acd-0768-4c09-b7de-20e90f957c04 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:25:08.719493 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-850d9acd-0768-4c09-b7de-20e90f957c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:25:08.728591 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 696/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 21:25:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:25:08.885208 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643fad7-13ef-4384-8da1-abb9de625729 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:25:08.902033 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643fad7-13ef-4384-8da1-abb9de625729 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:25:08.911259 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e715276-0b2f-4726-989b-9c148c0c06fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:25:08.912313 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e715276-0b2f-4726-989b-9c148c0c06fa service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:08.912801 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e715276-0b2f-4726-989b-9c148c0c06fa service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:08.913274 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e715276-0b2f-4726-989b-9c148c0c06fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:08.928596 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e715276-0b2f-4726-989b-9c148c0c06fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:25:08.931594 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e715276-0b2f-4726-989b-9c148c0c06fa service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:25:08.950502 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 729/1425] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:25:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:25:10.135410 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b0d3f-f650-43d4-b32a-ba3a1d9af014 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:10.136246 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b0d3f-f650-43d4-b32a-ba3a1d9af014 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:10.136246 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b0d3f-f650-43d4-b32a-ba3a1d9af014 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:10.137957 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 697/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 21:25:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:25:10.143843 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994972a8-66c0-4740-b8a8-37d9437c3aae None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:10.144417 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994972a8-66c0-4740-b8a8-37d9437c3aae None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:10.144417 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994972a8-66c0-4740-b8a8-37d9437c3aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:10.165996 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-994972a8-66c0-4740-b8a8-37d9437c3aae None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:25:10.167437 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-994972a8-66c0-4740-b8a8-37d9437c3aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:25:10.176196 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 730/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 21:25:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:25:10.326806 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b0d3f-f650-43d4-b32a-ba3a1d9af014 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:25:10.342626 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b0d3f-f650-43d4-b32a-ba3a1d9af014 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:25:10.348311 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75166ee8-3814-4bae-a8cc-554ab51bce5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:25:10.348976 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75166ee8-3814-4bae-a8cc-554ab51bce5f service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:10.349265 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75166ee8-3814-4bae-a8cc-554ab51bce5f service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:10.349535 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75166ee8-3814-4bae-a8cc-554ab51bce5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:10.359840 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75166ee8-3814-4bae-a8cc-554ab51bce5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:25:10.361522 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75166ee8-3814-4bae-a8cc-554ab51bce5f service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:25:10.373293 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 698/1428] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:25:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:25:11.286979 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb12b3-e03d-40c9-b901-f66ee878e79c None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:11.287794 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb12b3-e03d-40c9-b901-f66ee878e79c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:11.287794 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb12b3-e03d-40c9-b901-f66ee878e79c None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:11.289044 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 731/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 21:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:25:11.295843 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48888724-80cc-4ee6-a447-5ba75098cf4b None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:11.296478 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48888724-80cc-4ee6-a447-5ba75098cf4b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:11.296478 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48888724-80cc-4ee6-a447-5ba75098cf4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:11.317687 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-48888724-80cc-4ee6-a447-5ba75098cf4b None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:25:11.318963 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-48888724-80cc-4ee6-a447-5ba75098cf4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:25:11.329286 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 699/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 21:25:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:25:11.488947 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb12b3-e03d-40c9-b901-f66ee878e79c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:25:11.505032 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb12b3-e03d-40c9-b901-f66ee878e79c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:25:11.510637 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f351bdb-65e0-42dd-9d09-13ac9cd2de28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:25:11.511316 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f351bdb-65e0-42dd-9d09-13ac9cd2de28 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:11.511602 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f351bdb-65e0-42dd-9d09-13ac9cd2de28 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:11.511885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f351bdb-65e0-42dd-9d09-13ac9cd2de28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:11.521271 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f351bdb-65e0-42dd-9d09-13ac9cd2de28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:25:11.522907 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f351bdb-65e0-42dd-9d09-13ac9cd2de28 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:25:11.538250 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 732/1431] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:25:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:25:12.444137 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599204d-d297-4a25-a902-e646310bfb0e None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:12.444811 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599204d-d297-4a25-a902-e646310bfb0e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:12.444951 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599204d-d297-4a25-a902-e646310bfb0e None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:12.446584 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 700/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 21:25:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:25:12.452191 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34acb5eb-8f9c-4ac4-9eb3-2e1447da86f0 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:12.452723 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34acb5eb-8f9c-4ac4-9eb3-2e1447da86f0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:12.452894 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34acb5eb-8f9c-4ac4-9eb3-2e1447da86f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:12.472380 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-34acb5eb-8f9c-4ac4-9eb3-2e1447da86f0 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:25:12.473614 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-34acb5eb-8f9c-4ac4-9eb3-2e1447da86f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:25:12.481848 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 733/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 21:25:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:25:12.631971 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599204d-d297-4a25-a902-e646310bfb0e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:25:12.647668 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599204d-d297-4a25-a902-e646310bfb0e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:25:12.653561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b2e9d-81ce-48fa-ba5d-5b74808ae459 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:25:12.654302 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b2e9d-81ce-48fa-ba5d-5b74808ae459 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:12.654660 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b2e9d-81ce-48fa-ba5d-5b74808ae459 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:12.654919 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b2e9d-81ce-48fa-ba5d-5b74808ae459 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:12.664756 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b2e9d-81ce-48fa-ba5d-5b74808ae459 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:25:12.666482 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b2e9d-81ce-48fa-ba5d-5b74808ae459 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:25:12.677910 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 701/1434] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:25:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:25:13.650039 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a40d913-eee5-428a-b849-08312fa96893 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:13.650736 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a40d913-eee5-428a-b849-08312fa96893 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:13.650736 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a40d913-eee5-428a-b849-08312fa96893 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:13.652397 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 734/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 21:25:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:25:13.658970 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da347f8d-a45a-4105-8ea4-fb1c2ee4e2a5 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:13.659559 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da347f8d-a45a-4105-8ea4-fb1c2ee4e2a5 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:13.659559 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da347f8d-a45a-4105-8ea4-fb1c2ee4e2a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:13.679801 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-da347f8d-a45a-4105-8ea4-fb1c2ee4e2a5 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:25:13.681049 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-da347f8d-a45a-4105-8ea4-fb1c2ee4e2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:25:13.689864 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 702/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 21:25:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:25:13.846774 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a40d913-eee5-428a-b849-08312fa96893 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:25:13.864732 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a40d913-eee5-428a-b849-08312fa96893 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:25:13.871273 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736b92b-36f1-4fc7-a780-f98b2621c620 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:25:13.872043 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736b92b-36f1-4fc7-a780-f98b2621c620 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:13.872600 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736b92b-36f1-4fc7-a780-f98b2621c620 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:13.872600 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736b92b-36f1-4fc7-a780-f98b2621c620 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:13.882722 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3736b92b-36f1-4fc7-a780-f98b2621c620 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:25:13.884350 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3736b92b-36f1-4fc7-a780-f98b2621c620 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:25:13.896189 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 735/1437] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:25:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:25:14.807953 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9defb8a-1b36-475c-b343-f3988ebc7905 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:14.808540 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9defb8a-1b36-475c-b343-f3988ebc7905 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:14.808540 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9defb8a-1b36-475c-b343-f3988ebc7905 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:14.810026 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 703/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 21:25:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:25:14.816082 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b7532-d27e-46f4-b583-855be65256c5 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:14.816686 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b7532-d27e-46f4-b583-855be65256c5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:14.816686 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b7532-d27e-46f4-b583-855be65256c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:14.836423 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b55b7532-d27e-46f4-b583-855be65256c5 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:25:14.837655 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b55b7532-d27e-46f4-b583-855be65256c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:25:14.847083 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 736/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 21:25:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:25:15.002461 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9defb8a-1b36-475c-b343-f3988ebc7905 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:25:15.019377 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9defb8a-1b36-475c-b343-f3988ebc7905 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:25:15.025996 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a66bdbe-82fa-433e-831e-af2567a47650 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:25:15.026714 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66bdbe-82fa-433e-831e-af2567a47650 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:15.026997 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66bdbe-82fa-433e-831e-af2567a47650 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:15.027935 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66bdbe-82fa-433e-831e-af2567a47650 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:15.039124 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a66bdbe-82fa-433e-831e-af2567a47650 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:25:15.040508 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a66bdbe-82fa-433e-831e-af2567a47650 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:25:15.053277 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 704/1440] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:25:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:25:17.312578 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880fc82-9e95-447f-b77f-3c58f3c1ccfb None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:17.313144 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880fc82-9e95-447f-b77f-3c58f3c1ccfb None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:17.313144 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880fc82-9e95-447f-b77f-3c58f3c1ccfb None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:17.314447 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 737/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 21:25:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:25:17.319937 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25555a77-be81-4d52-8ea5-5c02ea702101 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:17.320408 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25555a77-be81-4d52-8ea5-5c02ea702101 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:17.320504 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25555a77-be81-4d52-8ea5-5c02ea702101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:17.343443 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-25555a77-be81-4d52-8ea5-5c02ea702101 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:25:17.344660 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-25555a77-be81-4d52-8ea5-5c02ea702101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:25:17.353938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 705/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 21:25:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:25:17.392404 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3880fc82-9e95-447f-b77f-3c58f3c1ccfb None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:25:17.404399 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3880fc82-9e95-447f-b77f-3c58f3c1ccfb None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:25:17.408130 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd0c48-06e4-4b51-aa70-0946c7c19883 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:25:17.408785 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd0c48-06e4-4b51-aa70-0946c7c19883 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:17.409066 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd0c48-06e4-4b51-aa70-0946c7c19883 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:17.409342 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd0c48-06e4-4b51-aa70-0946c7c19883 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:17.419111 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fd0c48-06e4-4b51-aa70-0946c7c19883 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:25:17.420581 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fd0c48-06e4-4b51-aa70-0946c7c19883 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:25:17.430585 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 738/1443] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:25:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:25:19.959631 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb951e-8869-4bb2-8641-8ba7c6ea050d None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:19.959631 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb951e-8869-4bb2-8641-8ba7c6ea050d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:19.960393 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb951e-8869-4bb2-8641-8ba7c6ea050d None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:19.961190 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 706/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 21:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:25:19.967434 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c55494-75a2-454e-947b-490672ba772d None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:19.967928 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c55494-75a2-454e-947b-490672ba772d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:19.967928 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c55494-75a2-454e-947b-490672ba772d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:19.989978 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-75c55494-75a2-454e-947b-490672ba772d None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:25:19.991232 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-75c55494-75a2-454e-947b-490672ba772d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:25:19.999741 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 739/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 21:25:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:25:20.039701 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb951e-8869-4bb2-8641-8ba7c6ea050d None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:25:20.052054 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb951e-8869-4bb2-8641-8ba7c6ea050d None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:25:20.056029 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d062269-b1f5-4f33-a121-151d40dde3b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:25:20.056740 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d062269-b1f5-4f33-a121-151d40dde3b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:20.057104 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d062269-b1f5-4f33-a121-151d40dde3b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:20.057385 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d062269-b1f5-4f33-a121-151d40dde3b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:20.067820 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d062269-b1f5-4f33-a121-151d40dde3b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:25:20.069290 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d062269-b1f5-4f33-a121-151d40dde3b8 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:25:20.080193 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 707/1446] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:25:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:25:22.695853 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6b38f-28b3-418b-8ae0-1ea32951b590 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:22.696487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6b38f-28b3-418b-8ae0-1ea32951b590 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:22.696487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6b38f-28b3-418b-8ae0-1ea32951b590 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:22.697691 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 740/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 21:25:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:25:22.703773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86981950-8a40-4f8d-9831-c88b7cf88e48 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:22.704319 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86981950-8a40-4f8d-9831-c88b7cf88e48 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:22.704453 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86981950-8a40-4f8d-9831-c88b7cf88e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:22.725345 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-86981950-8a40-4f8d-9831-c88b7cf88e48 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:25:22.726616 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-86981950-8a40-4f8d-9831-c88b7cf88e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:25:22.737128 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 708/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 21:25:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:25:22.778543 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6b38f-28b3-418b-8ae0-1ea32951b590 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:25:22.793560 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6b38f-28b3-418b-8ae0-1ea32951b590 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:25:22.797778 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161779b7-02a9-49e0-b3e4-6a2602e12e73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:25:22.798877 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161779b7-02a9-49e0-b3e4-6a2602e12e73 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:22.799421 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161779b7-02a9-49e0-b3e4-6a2602e12e73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:22.799739 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161779b7-02a9-49e0-b3e4-6a2602e12e73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:22.810442 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161779b7-02a9-49e0-b3e4-6a2602e12e73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:25:22.812012 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161779b7-02a9-49e0-b3e4-6a2602e12e73 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:25:22.822401 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 741/1449] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:25:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:25:25.273203 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa6747-7565-4b67-a5ab-4ce58be290d0 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:25.273805 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa6747-7565-4b67-a5ab-4ce58be290d0 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:25.273805 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa6747-7565-4b67-a5ab-4ce58be290d0 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:25.275102 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 709/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 21:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:25:25.281554 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1208dc-d589-4bbc-a0ae-4fed5574c6ff None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:25.282145 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1208dc-d589-4bbc-a0ae-4fed5574c6ff None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:25.282145 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1208dc-d589-4bbc-a0ae-4fed5574c6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:25.301242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-7a1208dc-d589-4bbc-a0ae-4fed5574c6ff None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:25:25.302631 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7a1208dc-d589-4bbc-a0ae-4fed5574c6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:25:25.311096 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 742/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 21:25:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:25:25.351911 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa6747-7565-4b67-a5ab-4ce58be290d0 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:25:25.354408 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-fbaa6747-7565-4b67-a5ab-4ce58be290d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:25:25.369369 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa6747-7565-4b67-a5ab-4ce58be290d0 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:25:25.372957 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac038d1-acf3-487d-82ba-b041b723ac20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:25:25.373609 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac038d1-acf3-487d-82ba-b041b723ac20 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:25.373886 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac038d1-acf3-487d-82ba-b041b723ac20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:25.374158 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac038d1-acf3-487d-82ba-b041b723ac20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:25.383832 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac038d1-acf3-487d-82ba-b041b723ac20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:25:25.385281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac038d1-acf3-487d-82ba-b041b723ac20 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:25:25.396886 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 710/1452] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:25:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:25:27.805541 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ad032-ffca-4a63-af34-e568ff07129e None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:27.806270 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ad032-ffca-4a63-af34-e568ff07129e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:27.806270 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ad032-ffca-4a63-af34-e568ff07129e None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:27.807737 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 743/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 21:25:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:25:27.813922 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731d2ee-64b2-43f2-bcae-6f3408c53df7 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:27.814545 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731d2ee-64b2-43f2-bcae-6f3408c53df7 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:27.814545 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731d2ee-64b2-43f2-bcae-6f3408c53df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:27.833864 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-a731d2ee-64b2-43f2-bcae-6f3408c53df7 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:25:27.835098 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a731d2ee-64b2-43f2-bcae-6f3408c53df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:25:27.844360 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 711/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 21:25:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:25:27.882678 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ad032-ffca-4a63-af34-e568ff07129e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:25:27.894090 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ad032-ffca-4a63-af34-e568ff07129e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:25:27.897725 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37ee0d-aa42-47b1-99e8-a6954f6eef26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:25:27.898410 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37ee0d-aa42-47b1-99e8-a6954f6eef26 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:27.898691 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37ee0d-aa42-47b1-99e8-a6954f6eef26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:27.898967 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37ee0d-aa42-47b1-99e8-a6954f6eef26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:27.908741 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d37ee0d-aa42-47b1-99e8-a6954f6eef26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:25:27.910180 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d37ee0d-aa42-47b1-99e8-a6954f6eef26 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:25:27.919712 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 744/1455] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:25:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:25:30.350735 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e28c145-3888-42be-8c23-e8732ecb5614 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:30.351604 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e28c145-3888-42be-8c23-e8732ecb5614 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:30.351604 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e28c145-3888-42be-8c23-e8732ecb5614 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:30.352699 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 712/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 21:25:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:25:30.359850 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b217d-cd13-480f-923b-d1b8beab258c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:30.360422 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b217d-cd13-480f-923b-d1b8beab258c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:30.360422 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b217d-cd13-480f-923b-d1b8beab258c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:30.366818 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=70 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:25:30.381503 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-086b217d-cd13-480f-923b-d1b8beab258c None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:25:30.382778 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-086b217d-cd13-480f-923b-d1b8beab258c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:25:30.391205 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 745/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 21:25:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:25:30.440227 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e28c145-3888-42be-8c23-e8732ecb5614 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:25:30.446285 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=64 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:25:30.453920 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e28c145-3888-42be-8c23-e8732ecb5614 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:25:30.457907 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fccea9f-4a95-4b0a-9ea3-56ff91e2dd74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:25:30.458571 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fccea9f-4a95-4b0a-9ea3-56ff91e2dd74 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:30.458848 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fccea9f-4a95-4b0a-9ea3-56ff91e2dd74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:30.459120 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fccea9f-4a95-4b0a-9ea3-56ff91e2dd74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:30.474072 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fccea9f-4a95-4b0a-9ea3-56ff91e2dd74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:25:30.476507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fccea9f-4a95-4b0a-9ea3-56ff91e2dd74 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:25:30.493493 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 713/1458] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:25:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:25:32.867567 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7caea-c553-4a9c-80ff-3d2ffbc9128e None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:32.868142 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7caea-c553-4a9c-80ff-3d2ffbc9128e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:32.868142 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7caea-c553-4a9c-80ff-3d2ffbc9128e None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:32.869592 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 746/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 21:25:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:25:32.876917 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad3c56-7b16-4ffd-b33f-12e628152dfc None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:32.877779 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad3c56-7b16-4ffd-b33f-12e628152dfc None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:32.877779 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad3c56-7b16-4ffd-b33f-12e628152dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:32.897652 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-cbad3c56-7b16-4ffd-b33f-12e628152dfc None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:25:32.898916 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-cbad3c56-7b16-4ffd-b33f-12e628152dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:25:32.907632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 714/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 21:25:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:25:32.945018 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca7caea-c553-4a9c-80ff-3d2ffbc9128e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:25:32.957385 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca7caea-c553-4a9c-80ff-3d2ffbc9128e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:25:32.960991 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ed07e-0adc-41d1-a8e2-f55ebc49d50a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:25:32.961650 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ed07e-0adc-41d1-a8e2-f55ebc49d50a service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:32.961931 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ed07e-0adc-41d1-a8e2-f55ebc49d50a service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:32.962204 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ed07e-0adc-41d1-a8e2-f55ebc49d50a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:32.972564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ed07e-0adc-41d1-a8e2-f55ebc49d50a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:25:32.974041 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ed07e-0adc-41d1-a8e2-f55ebc49d50a service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:25:32.984914 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 747/1461] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:25:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:25:35.338619 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14242b16-074f-4fde-9644-aa1c5d881904 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:35.339229 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14242b16-074f-4fde-9644-aa1c5d881904 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:35.339229 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14242b16-074f-4fde-9644-aa1c5d881904 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:35.340504 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 715/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 21:25:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:25:35.346954 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbad652-eba5-4cae-9cdf-400611419d06 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:35.347737 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbad652-eba5-4cae-9cdf-400611419d06 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:35.347939 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbad652-eba5-4cae-9cdf-400611419d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:35.368683 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-afbad652-eba5-4cae-9cdf-400611419d06 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:25:35.370061 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-afbad652-eba5-4cae-9cdf-400611419d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:25:35.383984 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 748/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 21:25:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:25:35.423257 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14242b16-074f-4fde-9644-aa1c5d881904 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:25:35.434838 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14242b16-074f-4fde-9644-aa1c5d881904 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:25:35.438852 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f7153-2b16-4e5a-a077-b2705a0ab2ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:25:35.439442 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f7153-2b16-4e5a-a077-b2705a0ab2ab service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:35.439722 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f7153-2b16-4e5a-a077-b2705a0ab2ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:35.440000 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f7153-2b16-4e5a-a077-b2705a0ab2ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:35.450502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f7153-2b16-4e5a-a077-b2705a0ab2ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:25:35.451762 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f7153-2b16-4e5a-a077-b2705a0ab2ab service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:25:35.461694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 716/1464] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:25:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:25:37.836421 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702e6c1-5c1e-4ed5-822c-e42eb5c0b207 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:37.837004 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702e6c1-5c1e-4ed5-822c-e42eb5c0b207 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:37.837138 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702e6c1-5c1e-4ed5-822c-e42eb5c0b207 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:37.838802 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 749/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 21:25:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:25:37.845779 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6162c5-6ddd-43e3-af41-ee1692b846ef None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:37.846301 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6162c5-6ddd-43e3-af41-ee1692b846ef None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:37.846626 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6162c5-6ddd-43e3-af41-ee1692b846ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:37.867622 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-7f6162c5-6ddd-43e3-af41-ee1692b846ef None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:25:37.868882 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7f6162c5-6ddd-43e3-af41-ee1692b846ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:25:37.879209 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 717/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 21:25:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:25:37.916327 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d702e6c1-5c1e-4ed5-822c-e42eb5c0b207 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:25:37.928820 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d702e6c1-5c1e-4ed5-822c-e42eb5c0b207 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:25:37.932911 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f332e669-dfd0-4020-beb5-c613d9c0e23d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:25:37.933565 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f332e669-dfd0-4020-beb5-c613d9c0e23d service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:37.933848 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f332e669-dfd0-4020-beb5-c613d9c0e23d service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:37.934163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f332e669-dfd0-4020-beb5-c613d9c0e23d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:37.944816 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f332e669-dfd0-4020-beb5-c613d9c0e23d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:25:37.946515 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f332e669-dfd0-4020-beb5-c613d9c0e23d service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:25:37.957192 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 750/1467] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:25:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:25:40.274991 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985af3b8-4197-4674-96b1-72ca7fdf5634 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:40.275661 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985af3b8-4197-4674-96b1-72ca7fdf5634 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:40.275661 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985af3b8-4197-4674-96b1-72ca7fdf5634 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:40.276927 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 718/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 21:25:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:25:40.283200 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb3ea0-3698-4201-b070-8f93a0d11ba7 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:40.283874 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb3ea0-3698-4201-b070-8f93a0d11ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:40.283874 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb3ea0-3698-4201-b070-8f93a0d11ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:40.304512 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-43bb3ea0-3698-4201-b070-8f93a0d11ba7 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:25:40.305821 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-43bb3ea0-3698-4201-b070-8f93a0d11ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:25:40.316164 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 751/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 21:25:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:25:40.357436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985af3b8-4197-4674-96b1-72ca7fdf5634 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:25:40.370517 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985af3b8-4197-4674-96b1-72ca7fdf5634 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:25:40.374915 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad03bcf-04ec-4681-8cf1-9e42c677d0fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:25:40.375594 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad03bcf-04ec-4681-8cf1-9e42c677d0fe service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:25:40.375983 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad03bcf-04ec-4681-8cf1-9e42c677d0fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:25:40.376165 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad03bcf-04ec-4681-8cf1-9e42c677d0fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:25:40.388922 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad03bcf-04ec-4681-8cf1-9e42c677d0fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:25:40.392009 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad03bcf-04ec-4681-8cf1-9e42c677d0fe service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:25:40.411938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 719/1470] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:25:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:25:50.297745 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=20 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:25:50.403051 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=23 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:26:14.168878 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b745e2-4fc8-45bb-8229-1815af3b3253 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:26:14.169504 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b745e2-4fc8-45bb-8229-1815af3b3253 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:26:14.169504 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b745e2-4fc8-45bb-8229-1815af3b3253 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:26:14.170844 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 752/1471] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:26:14.176570 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a6edb-9876-4a62-8eba-96eb504ea230 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:26:14.177006 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a6edb-9876-4a62-8eba-96eb504ea230 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:26:14.177168 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a6edb-9876-4a62-8eba-96eb504ea230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:26:14.195518 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-212a6edb-9876-4a62-8eba-96eb504ea230 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:26:14.196808 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-212a6edb-9876-4a62-8eba-96eb504ea230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:26:14.205448 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 720/1472] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:26:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:26:14.254271 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e1ff1-bee4-445e-ac36-04e664d1a18a None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:26:14.254572 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e1ff1-bee4-445e-ac36-04e664d1a18a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:26:14.254850 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e1ff1-bee4-445e-ac36-04e664d1a18a None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:26:14.256255 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 753/1473] 10.222.0.65 () {54 vars in 962 bytes} [Thu May 19 21:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:26:14.261471 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f9936-874e-4889-9ca6-fb321b4bb08d None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:26:14.261985 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f9936-874e-4889-9ca6-fb321b4bb08d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:26:14.262329 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f9936-874e-4889-9ca6-fb321b4bb08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:26:14.282029 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-cc6f9936-874e-4889-9ca6-fb321b4bb08d None None] MFA Rules not processed for user `ea874ca4088d4e6c83c9d0497323cf6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:26:14.283298 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-cc6f9936-874e-4889-9ca6-fb321b4bb08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:26:14.290047 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 721/1474] 10.222.0.65 () {58 vars in 1059 bytes} [Thu May 19 21:26:14 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:26:14.297740 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e1ff1-bee4-445e-ac36-04e664d1a18a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:26:14.309473 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e1ff1-bee4-445e-ac36-04e664d1a18a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:26:14.313486 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c610a715-aad3-457b-b87c-654e02e11c49 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea874ca4088d4e6c83c9d0497323cf6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:26:14.314148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610a715-aad3-457b-b87c-654e02e11c49 service glance] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:26:14.314462 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610a715-aad3-457b-b87c-654e02e11c49 service glance] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:26:14.314732 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610a715-aad3-457b-b87c-654e02e11c49 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:26:14.324584 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c610a715-aad3-457b-b87c-654e02e11c49 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:26:14.326034 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c610a715-aad3-457b-b87c-654e02e11c49 service glance] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:26:14.336249 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 754/1475] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:26:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:26:16.753158 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b68af-9f58-4d1d-a86e-8573b2f03c90 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:26:16.753717 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b68af-9f58-4d1d-a86e-8573b2f03c90 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:26:16.753717 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b68af-9f58-4d1d-a86e-8573b2f03c90 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:26:16.755713 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 722/1476] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:26:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:26:16.763478 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1354a625-9995-4a02-89fa-acb4ce4f4a1b None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:26:16.763921 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1354a625-9995-4a02-89fa-acb4ce4f4a1b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:26:16.764023 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1354a625-9995-4a02-89fa-acb4ce4f4a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:26:16.782493 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-1354a625-9995-4a02-89fa-acb4ce4f4a1b None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:26:16.783706 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1354a625-9995-4a02-89fa-acb4ce4f4a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:26:16.792234 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 755/1477] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:26:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:26:16.815490 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6b68af-9f58-4d1d-a86e-8573b2f03c90 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:26:16.829439 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6b68af-9f58-4d1d-a86e-8573b2f03c90 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:26:16.834443 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce4206f-8422-43d1-b4b0-1ca29a1a2c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:26:16.835104 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4206f-8422-43d1-b4b0-1ca29a1a2c6e admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:26:16.835384 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4206f-8422-43d1-b4b0-1ca29a1a2c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:26:16.835718 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4206f-8422-43d1-b4b0-1ca29a1a2c6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:26:16.836480 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce4206f-8422-43d1-b4b0-1ca29a1a2c6e admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:26:16.838294 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: WARNING py.warnings [None req-0ce4206f-8422-43d1-b4b0-1ca29a1a2c6e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 21:26:16.838294 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: warnings.warn(msg) May 19 21:26:16.838294 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: May 19 21:26:16.839431 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce4206f-8422-43d1-b4b0-1ca29a1a2c6e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:26:16.862460 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 723/1478] 10.222.0.65 () {56 vars in 1108 bytes} [Thu May 19 21:26:16 2022] GET /identity/v3/projects => generated 2553 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:26:19.127981 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03990ac-8a0f-40ba-b676-0958f6a2d215 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:26:19.128517 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03990ac-8a0f-40ba-b676-0958f6a2d215 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:26:19.128517 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03990ac-8a0f-40ba-b676-0958f6a2d215 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:26:19.129975 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 756/1479] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:26:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:26:19.138434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612f11c-df94-4904-95a5-483971f6783a None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:26:19.138976 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612f11c-df94-4904-95a5-483971f6783a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:26:19.139742 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612f11c-df94-4904-95a5-483971f6783a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:26:19.172441 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-e612f11c-df94-4904-95a5-483971f6783a None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:26:19.173653 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e612f11c-df94-4904-95a5-483971f6783a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:26:19.182015 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 724/1480] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:26:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:26:19.226404 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03990ac-8a0f-40ba-b676-0958f6a2d215 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:26:19.242162 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03990ac-8a0f-40ba-b676-0958f6a2d215 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:26:19.248055 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ed2c8-c296-49a4-bf01-0425c212d287 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:26:19.248702 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ed2c8-c296-49a4-bf01-0425c212d287 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:26:19.248985 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ed2c8-c296-49a4-bf01-0425c212d287 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:26:19.249243 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ed2c8-c296-49a4-bf01-0425c212d287 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:26:19.257790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ed2c8-c296-49a4-bf01-0425c212d287 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:26:19.259410 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ed2c8-c296-49a4-bf01-0425c212d287 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:26:19.269185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 757/1481] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:26:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:26:21.716057 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f0bf0-78f4-4c3d-a166-e7b3bc95c369 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:26:21.716952 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f0bf0-78f4-4c3d-a166-e7b3bc95c369 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:26:21.716952 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f0bf0-78f4-4c3d-a166-e7b3bc95c369 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:26:21.717996 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 725/1482] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:26:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:26:21.724686 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a5d46-a1b7-4407-b744-bd2edce12800 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:26:21.725198 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a5d46-a1b7-4407-b744-bd2edce12800 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:26:21.725198 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a5d46-a1b7-4407-b744-bd2edce12800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:26:21.745607 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c44a5d46-a1b7-4407-b744-bd2edce12800 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:26:21.746986 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c44a5d46-a1b7-4407-b744-bd2edce12800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:26:21.755772 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 758/1483] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:26:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:26:21.796631 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f0bf0-78f4-4c3d-a166-e7b3bc95c369 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:26:21.813480 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f0bf0-78f4-4c3d-a166-e7b3bc95c369 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:26:21.818114 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a13c4-684e-474b-ba5e-d991456585e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:26:21.818791 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a13c4-684e-474b-ba5e-d991456585e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:26:21.819124 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a13c4-684e-474b-ba5e-d991456585e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:26:21.819408 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a13c4-684e-474b-ba5e-d991456585e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:26:21.830065 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a13c4-684e-474b-ba5e-d991456585e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:26:21.831558 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a13c4-684e-474b-ba5e-d991456585e0 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:26:21.841653 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 726/1484] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:26:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:26:24.167116 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cb123-501e-4175-8899-e46ff2a88192 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:26:24.167753 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cb123-501e-4175-8899-e46ff2a88192 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:26:24.167753 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cb123-501e-4175-8899-e46ff2a88192 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:26:24.169174 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 759/1485] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:26:24.175162 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb71bcb-a13f-4d8f-ad5b-cab8801c78d5 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:26:24.175628 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb71bcb-a13f-4d8f-ad5b-cab8801c78d5 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:26:24.175734 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb71bcb-a13f-4d8f-ad5b-cab8801c78d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:26:24.195301 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-ecb71bcb-a13f-4d8f-ad5b-cab8801c78d5 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:26:24.196546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ecb71bcb-a13f-4d8f-ad5b-cab8801c78d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:26:24.206383 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 727/1486] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:26:24.244727 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8cb123-501e-4175-8899-e46ff2a88192 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:26:24.256666 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8cb123-501e-4175-8899-e46ff2a88192 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:26:24.260546 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b18263-e242-476d-aa50-01b6f56ef15d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:26:24.261200 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b18263-e242-476d-aa50-01b6f56ef15d service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:26:24.261481 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b18263-e242-476d-aa50-01b6f56ef15d service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:26:24.262278 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b18263-e242-476d-aa50-01b6f56ef15d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:26:24.271513 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b18263-e242-476d-aa50-01b6f56ef15d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:26:24.272938 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b18263-e242-476d-aa50-01b6f56ef15d service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:26:24.282618 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 760/1487] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:26:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:26:26.683543 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a639e07-e856-44f5-96f4-c3551ce79d46 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:26:26.684086 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a639e07-e856-44f5-96f4-c3551ce79d46 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:26:26.684086 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a639e07-e856-44f5-96f4-c3551ce79d46 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:26:26.686009 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 728/1488] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:26:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:26:26.691659 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e592086-7c7d-428b-902d-5885f90ea059 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:26:26.692162 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e592086-7c7d-428b-902d-5885f90ea059 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:26:26.692162 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e592086-7c7d-428b-902d-5885f90ea059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:26:26.711465 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-6e592086-7c7d-428b-902d-5885f90ea059 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:26:26.712690 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6e592086-7c7d-428b-902d-5885f90ea059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:26:26.722362 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 761/1489] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:26:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:26:26.763675 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a639e07-e856-44f5-96f4-c3551ce79d46 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:26:26.776271 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a639e07-e856-44f5-96f4-c3551ce79d46 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:26:26.780794 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ab242-477f-4f9f-a146-39d95752af0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:26:26.781445 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ab242-477f-4f9f-a146-39d95752af0c service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:26:26.781725 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ab242-477f-4f9f-a146-39d95752af0c service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:26:26.782000 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ab242-477f-4f9f-a146-39d95752af0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:26:26.792000 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932ab242-477f-4f9f-a146-39d95752af0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:26:26.793817 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932ab242-477f-4f9f-a146-39d95752af0c service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:26:26.803712 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 729/1490] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:26:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:26:29.272941 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee389c-4e6a-4cef-ab4e-2d71ccb1f992 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:26:29.273510 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee389c-4e6a-4cef-ab4e-2d71ccb1f992 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:26:29.273510 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee389c-4e6a-4cef-ab4e-2d71ccb1f992 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:26:29.274791 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 762/1491] 10.222.0.65 () {54 vars in 926 bytes} [Thu May 19 21:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:26:29.280612 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f65df2-27a6-4166-80ab-ebe9d031596d None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:26:29.281065 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f65df2-27a6-4166-80ab-ebe9d031596d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:26:29.281180 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f65df2-27a6-4166-80ab-ebe9d031596d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:26:29.300015 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-56f65df2-27a6-4166-80ab-ebe9d031596d None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:26:29.301225 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-56f65df2-27a6-4166-80ab-ebe9d031596d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:26:29.310670 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 730/1492] 10.222.0.65 () {58 vars in 1023 bytes} [Thu May 19 21:26:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:26:29.350011 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee389c-4e6a-4cef-ab4e-2d71ccb1f992 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:26:29.362871 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee389c-4e6a-4cef-ab4e-2d71ccb1f992 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:26:29.367170 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bbb47f-ae99-40dc-b4e4-61a1536f3edf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:26:29.367825 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bbb47f-ae99-40dc-b4e4-61a1536f3edf service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:26:29.368109 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bbb47f-ae99-40dc-b4e4-61a1536f3edf service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:26:29.368392 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bbb47f-ae99-40dc-b4e4-61a1536f3edf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:26:29.378470 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bbb47f-ae99-40dc-b4e4-61a1536f3edf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:26:29.379913 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bbb47f-ae99-40dc-b4e4-61a1536f3edf service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:26:29.389874 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 763/1493] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:26:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:26:39.294487 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=33 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:26:39.385349 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=33 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:28:20.464528 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f6609-58bd-4148-992c-cf341dfa2919 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:28:20.465267 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f6609-58bd-4148-992c-cf341dfa2919 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:28:20.465267 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f6609-58bd-4148-992c-cf341dfa2919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:28:20.485188 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-458f6609-58bd-4148-992c-cf341dfa2919 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:28:20.486539 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-458f6609-58bd-4148-992c-cf341dfa2919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:28:20.494992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 731/1494] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:28:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:28:20.504668 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5116f6-3e52-497b-ac38-04b3374355fd None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:28:20.505227 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5116f6-3e52-497b-ac38-04b3374355fd None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:28:20.505227 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5116f6-3e52-497b-ac38-04b3374355fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:28:20.523538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0a5116f6-3e52-497b-ac38-04b3374355fd None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:28:20.524937 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0a5116f6-3e52-497b-ac38-04b3374355fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:28:20.533339 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 764/1495] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:28:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:28:20.538224 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f6609-58bd-4148-992c-cf341dfa2919 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:28:20.549435 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f6609-58bd-4148-992c-cf341dfa2919 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:28:20.554406 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e8620-836a-4a0e-ad7c-96648d6bda20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:28:20.555078 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e8620-836a-4a0e-ad7c-96648d6bda20 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:28:20.555360 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e8620-836a-4a0e-ad7c-96648d6bda20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:28:20.555637 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e8620-836a-4a0e-ad7c-96648d6bda20 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:28:20.556809 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067e8620-836a-4a0e-ad7c-96648d6bda20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:28:20.557120 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067e8620-836a-4a0e-ad7c-96648d6bda20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:28:20.558158 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: WARNING py.warnings [None req-067e8620-836a-4a0e-ad7c-96648d6bda20 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 21:28:20.558158 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: warnings.warn(msg) May 19 21:28:20.558158 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: May 19 21:28:20.559091 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067e8620-836a-4a0e-ad7c-96648d6bda20 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:28:20.571480 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 732/1496] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:28:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:28:20.580044 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5116f6-3e52-497b-ac38-04b3374355fd None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:28:20.590288 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5116f6-3e52-497b-ac38-04b3374355fd None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:28:20.596144 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04e8d4-249b-444b-8983-2bf61f7f0553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:28:20.596816 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04e8d4-249b-444b-8983-2bf61f7f0553 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:28:20.597095 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04e8d4-249b-444b-8983-2bf61f7f0553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:28:20.597368 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04e8d4-249b-444b-8983-2bf61f7f0553 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:28:20.598072 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f04e8d4-249b-444b-8983-2bf61f7f0553 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:28:20.599931 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: WARNING py.warnings [None req-0f04e8d4-249b-444b-8983-2bf61f7f0553 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 21:28:20.599931 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: warnings.warn(msg) May 19 21:28:20.599931 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: May 19 21:28:20.601124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f04e8d4-249b-444b-8983-2bf61f7f0553 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:28:20.646423 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 765/1497] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:28:20 2022] POST /identity/v3/projects => generated 377 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:28:20.650952 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e8620-836a-4a0e-ad7c-96648d6bda20 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:28:20.664462 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e8620-836a-4a0e-ad7c-96648d6bda20 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:28:20.727125 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa3b09d-839f-48f5-91f1-0dceda81e9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:28:20.727800 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa3b09d-839f-48f5-91f1-0dceda81e9e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:28:20.728036 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa3b09d-839f-48f5-91f1-0dceda81e9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:28:20.728315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa3b09d-839f-48f5-91f1-0dceda81e9e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:28:20.729425 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa3b09d-839f-48f5-91f1-0dceda81e9e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:28:20.730506 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: WARNING py.warnings [None req-1fa3b09d-839f-48f5-91f1-0dceda81e9e3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 21:28:20.730506 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: warnings.warn(msg) May 19 21:28:20.730506 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: May 19 21:28:20.731757 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa3b09d-839f-48f5-91f1-0dceda81e9e3 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:28:20.754475 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 733/1498] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:28:20 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:28:20.758810 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04e8d4-249b-444b-8983-2bf61f7f0553 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:28:20.769437 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04e8d4-249b-444b-8983-2bf61f7f0553 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:28:20.774178 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210ba3c-8d9d-4991-9ba5-2538cb317db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:28:20.774872 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210ba3c-8d9d-4991-9ba5-2538cb317db3 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:28:20.775204 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210ba3c-8d9d-4991-9ba5-2538cb317db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:28:20.775513 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210ba3c-8d9d-4991-9ba5-2538cb317db3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:28:20.776244 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2210ba3c-8d9d-4991-9ba5-2538cb317db3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:28:20.777255 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: WARNING py.warnings [None req-2210ba3c-8d9d-4991-9ba5-2538cb317db3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 21:28:20.777255 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: warnings.warn(msg) May 19 21:28:20.777255 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: May 19 21:28:20.778120 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2210ba3c-8d9d-4991-9ba5-2538cb317db3 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:28:20.791435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 766/1499] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:28:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:28:20.795893 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa3b09d-839f-48f5-91f1-0dceda81e9e3 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:28:20.806182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa3b09d-839f-48f5-91f1-0dceda81e9e3 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:28:20.811145 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5af1b5-7f72-40fd-b16c-7361679fbac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:28:20.811806 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5af1b5-7f72-40fd-b16c-7361679fbac3 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:28:20.812087 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5af1b5-7f72-40fd-b16c-7361679fbac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:28:20.812359 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5af1b5-7f72-40fd-b16c-7361679fbac3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:28:20.813082 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5af1b5-7f72-40fd-b16c-7361679fbac3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:28:20.814086 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: WARNING py.warnings [None req-3e5af1b5-7f72-40fd-b16c-7361679fbac3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 21:28:20.814086 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: warnings.warn(msg) May 19 21:28:20.814086 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: May 19 21:28:20.815027 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5af1b5-7f72-40fd-b16c-7361679fbac3 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:28:20.829987 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 734/1500] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:28:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:28:20.834423 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210ba3c-8d9d-4991-9ba5-2538cb317db3 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:28:20.845704 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210ba3c-8d9d-4991-9ba5-2538cb317db3 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:28:20.850665 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34231a-25e7-4b18-ac80-d492a82d87b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:28:20.851579 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34231a-25e7-4b18-ac80-d492a82d87b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:28:20.851857 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34231a-25e7-4b18-ac80-d492a82d87b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:28:20.852132 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34231a-25e7-4b18-ac80-d492a82d87b1 admin admin] PATH_INFO: `/v3/projects/a6b428dd77ea4dc6bacf7851c72f9928/users/33c9b20c301a424aac3b40ce9445f445/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:28:20.873344 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec34231a-25e7-4b18-ac80-d492a82d87b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6b428dd77ea4dc6bacf7851c72f9928, user_id=33c9b20c301a424aac3b40ce9445f445, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:28:20.874426 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: WARNING py.warnings [None req-ec34231a-25e7-4b18-ac80-d492a82d87b1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 21:28:20.874426 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: warnings.warn(msg) May 19 21:28:20.874426 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: May 19 21:28:20.876356 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec34231a-25e7-4b18-ac80-d492a82d87b1 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:28:20.891347 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 767/1501] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:28:20 2022] PUT /identity/v3/projects/a6b428dd77ea4dc6bacf7851c72f9928/users/33c9b20c301a424aac3b40ce9445f445/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:28:20.903384 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94234d10-3ea4-416b-9b2a-4a1ddee6830b None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:28:20.903647 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94234d10-3ea4-416b-9b2a-4a1ddee6830b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:28:20.903944 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94234d10-3ea4-416b-9b2a-4a1ddee6830b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:28:20.933849 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-94234d10-3ea4-416b-9b2a-4a1ddee6830b None None] MFA Rules not processed for user `33c9b20c301a424aac3b40ce9445f445`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:28:20.935136 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-94234d10-3ea4-416b-9b2a-4a1ddee6830b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:28:21.006184 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 735/1502] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:28:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:28:21.020359 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48700003-808b-4323-a662-a72b93c6031a None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:28:21.020644 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48700003-808b-4323-a662-a72b93c6031a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:28:21.020923 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48700003-808b-4323-a662-a72b93c6031a None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:28:21.022374 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 768/1503] 10.222.0.65 () {54 vars in 963 bytes} [Thu May 19 21:28:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:28:21.028328 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae95be-edbe-41aa-abb2-eed5af9db828 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:28:21.028620 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae95be-edbe-41aa-abb2-eed5af9db828 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:28:21.028900 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae95be-edbe-41aa-abb2-eed5af9db828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:28:21.051188 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-22ae95be-edbe-41aa-abb2-eed5af9db828 None None] MFA Rules not processed for user `51852131d4e5448f8152a51159bd78e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:28:21.052469 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-22ae95be-edbe-41aa-abb2-eed5af9db828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:28:21.106091 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:28:21.114962 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 736/1504] 10.222.0.65 () {58 vars in 1060 bytes} [Thu May 19 21:28:21 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:28:21.127707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48700003-808b-4323-a662-a72b93c6031a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:28:21.140947 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48700003-808b-4323-a662-a72b93c6031a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:28:21.144634 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e7739-a53d-41e9-b7fe-8f9eaa522887 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:28:21.145294 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e7739-a53d-41e9-b7fe-8f9eaa522887 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:28:21.145572 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e7739-a53d-41e9-b7fe-8f9eaa522887 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:28:21.145844 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e7739-a53d-41e9-b7fe-8f9eaa522887 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:28:21.155542 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e7739-a53d-41e9-b7fe-8f9eaa522887 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:28:21.157045 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e7739-a53d-41e9-b7fe-8f9eaa522887 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:28:21.166516 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 769/1505] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:28:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:28:21.208984 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c551b-bc9c-4c07-8678-e9db7ba849fd None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:28:21.209272 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c551b-bc9c-4c07-8678-e9db7ba849fd None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:28:21.209552 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c551b-bc9c-4c07-8678-e9db7ba849fd None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:28:21.210987 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 737/1506] 10.222.0.65 () {52 vars in 829 bytes} [Thu May 19 21:28:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:28:21.214834 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e7739-a53d-41e9-b7fe-8f9eaa522887 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:28:21.284551 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e7739-a53d-41e9-b7fe-8f9eaa522887 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:28:21.289547 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace14a10-c178-4bb0-99da-cf85ba507c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:28:21.291088 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace14a10-c178-4bb0-99da-cf85ba507c4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:28:21.291378 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace14a10-c178-4bb0-99da-cf85ba507c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:28:21.291660 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace14a10-c178-4bb0-99da-cf85ba507c4f admin admin] PATH_INFO: `/v3/users/33c9b20c301a424aac3b40ce9445f445` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:28:21.292902 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace14a10-c178-4bb0-99da-cf85ba507c4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=33c9b20c301a424aac3b40ce9445f445)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:28:21.293931 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: WARNING py.warnings [None req-ace14a10-c178-4bb0-99da-cf85ba507c4f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 21:28:21.293931 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: warnings.warn(msg) May 19 21:28:21.293931 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: May 19 21:28:21.295198 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace14a10-c178-4bb0-99da-cf85ba507c4f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:28:21.365195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ace14a10-c178-4bb0-99da-cf85ba507c4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33c9b20c301a424aac3b40ce9445f445'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:28:21.385176 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ace14a10-c178-4bb0-99da-cf85ba507c4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33c9b20c301a424aac3b40ce9445f445'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:28:21.386082 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ace14a10-c178-4bb0-99da-cf85ba507c4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33c9b20c301a424aac3b40ce9445f445'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:28:21.386781 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ace14a10-c178-4bb0-99da-cf85ba507c4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33c9b20c301a424aac3b40ce9445f445'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:28:21.400636 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ace14a10-c178-4bb0-99da-cf85ba507c4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33c9b20c301a424aac3b40ce9445f445'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:28:21.404856 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:28:21.418879 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 770/1507] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:28:21 2022] DELETE /identity/v3/users/33c9b20c301a424aac3b40ce9445f445 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:28:21.423537 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df6758-bbdd-4b55-ae74-1b9a8c102e75 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:28:21.423906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df6758-bbdd-4b55-ae74-1b9a8c102e75 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:28:21.424145 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df6758-bbdd-4b55-ae74-1b9a8c102e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:28:21.443188 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-19df6758-bbdd-4b55-ae74-1b9a8c102e75 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:28:21.444396 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-19df6758-bbdd-4b55-ae74-1b9a8c102e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:28:21.508747 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 738/1508] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:28:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:28:21.518748 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace14a10-c178-4bb0-99da-cf85ba507c4f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:28:21.520403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ace14a10-c178-4bb0-99da-cf85ba507c4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:28:21.585467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace14a10-c178-4bb0-99da-cf85ba507c4f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:28:21.589468 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4720c457-663f-431c-9d92-e1b444e9e6cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:28:21.590137 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720c457-663f-431c-9d92-e1b444e9e6cf service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:28:21.590458 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720c457-663f-431c-9d92-e1b444e9e6cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:28:21.590737 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720c457-663f-431c-9d92-e1b444e9e6cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:28:21.600806 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4720c457-663f-431c-9d92-e1b444e9e6cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:28:21.602284 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4720c457-663f-431c-9d92-e1b444e9e6cf service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:28:21.612842 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 771/1509] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:28:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:28:21.952236 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df6758-bbdd-4b55-ae74-1b9a8c102e75 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:28:21.954561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-19df6758-bbdd-4b55-ae74-1b9a8c102e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:28:21.970112 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df6758-bbdd-4b55-ae74-1b9a8c102e75 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:28:21.975233 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea5afc2-d94c-49a9-837b-59c6d533f1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:28:21.976131 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5afc2-d94c-49a9-837b-59c6d533f1fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:28:21.976411 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5afc2-d94c-49a9-837b-59c6d533f1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:28:21.976687 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5afc2-d94c-49a9-837b-59c6d533f1fe admin admin] PATH_INFO: `/v3/projects/a6b428dd77ea4dc6bacf7851c72f9928` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:28:21.979067 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea5afc2-d94c-49a9-837b-59c6d533f1fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6b428dd77ea4dc6bacf7851c72f9928)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:28:21.979893 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: WARNING py.warnings [None req-3ea5afc2-d94c-49a9-837b-59c6d533f1fe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 21:28:21.979893 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: warnings.warn(msg) May 19 21:28:21.979893 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: May 19 21:28:21.981133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea5afc2-d94c-49a9-837b-59c6d533f1fe admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:28:22.051121 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-3ea5afc2-d94c-49a9-837b-59c6d533f1fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6b428dd77ea4dc6bacf7851c72f9928', 'request_id': 'req-3ea5afc2-d94c-49a9-837b-59c6d533f1fe'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:28:22.064737 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-3ea5afc2-d94c-49a9-837b-59c6d533f1fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6b428dd77ea4dc6bacf7851c72f9928', 'request_id': 'req-3ea5afc2-d94c-49a9-837b-59c6d533f1fe'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:28:22.076638 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-3ea5afc2-d94c-49a9-837b-59c6d533f1fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6b428dd77ea4dc6bacf7851c72f9928', 'request_id': 'req-3ea5afc2-d94c-49a9-837b-59c6d533f1fe'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:28:22.089571 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-3ea5afc2-d94c-49a9-837b-59c6d533f1fe admin admin] The token cache is being invalidate because project a6b428dd77ea4dc6bacf7851c72f9928 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:28:22.089916 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-3ea5afc2-d94c-49a9-837b-59c6d533f1fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:28:22.090781 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-3ea5afc2-d94c-49a9-837b-59c6d533f1fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:28:22.092455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 739/1510] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:28:21 2022] DELETE /identity/v3/projects/a6b428dd77ea4dc6bacf7851c72f9928 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:28:31.608345 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:28:32.085240 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:30:04.189322 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4ff16-5946-4916-9f9b-70081fed26cf None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:30:04.190144 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4ff16-5946-4916-9f9b-70081fed26cf None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:30:04.190144 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4ff16-5946-4916-9f9b-70081fed26cf None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:30:04.192280 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 772/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 21:30:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:30:04.199143 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7cb7b-4651-4fd9-801e-49bfb1896ede None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:30:04.200332 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7cb7b-4651-4fd9-801e-49bfb1896ede None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:30:04.200645 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7cb7b-4651-4fd9-801e-49bfb1896ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:30:04.221206 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-49c7cb7b-4651-4fd9-801e-49bfb1896ede None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:30:04.222612 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-49c7cb7b-4651-4fd9-801e-49bfb1896ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:30:04.307675 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 740/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 21:30:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:30:04.478624 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea4ff16-5946-4916-9f9b-70081fed26cf None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:30:04.481442 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6ea4ff16-5946-4916-9f9b-70081fed26cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:30:04.566866 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea4ff16-5946-4916-9f9b-70081fed26cf None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:30:04.572905 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06eefa-9840-4ac9-9782-455ea53822ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:30:04.573583 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06eefa-9840-4ac9-9782-455ea53822ae service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:30:04.573869 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06eefa-9840-4ac9-9782-455ea53822ae service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:30:04.574152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06eefa-9840-4ac9-9782-455ea53822ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:30:04.583404 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be06eefa-9840-4ac9-9782-455ea53822ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:30:04.585206 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be06eefa-9840-4ac9-9782-455ea53822ae service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:30:04.596558 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 773/1513] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:30:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:30:05.495907 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0a9f0-323b-4fb1-b899-59f492bb57c5 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:30:05.496823 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0a9f0-323b-4fb1-b899-59f492bb57c5 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:30:05.496823 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0a9f0-323b-4fb1-b899-59f492bb57c5 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:30:05.499172 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 741/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 21:30:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:30:05.506352 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893336c-1c77-45ba-b456-4c0d71f20c46 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:30:05.506811 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893336c-1c77-45ba-b456-4c0d71f20c46 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:30:05.506912 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893336c-1c77-45ba-b456-4c0d71f20c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:30:05.526916 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a893336c-1c77-45ba-b456-4c0d71f20c46 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:30:05.528117 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a893336c-1c77-45ba-b456-4c0d71f20c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:30:05.536854 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 774/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 21:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:30:05.687743 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee0a9f0-323b-4fb1-b899-59f492bb57c5 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:30:05.704829 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee0a9f0-323b-4fb1-b899-59f492bb57c5 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:30:05.713893 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187490dc-e78e-48ad-bca9-6937b3833c18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:30:05.714885 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187490dc-e78e-48ad-bca9-6937b3833c18 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:30:05.715238 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187490dc-e78e-48ad-bca9-6937b3833c18 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:30:05.715605 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187490dc-e78e-48ad-bca9-6937b3833c18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:30:05.731281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187490dc-e78e-48ad-bca9-6937b3833c18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:30:05.732937 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187490dc-e78e-48ad-bca9-6937b3833c18 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:30:05.743209 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 742/1516] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:30:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:30:05.982850 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367c72f-0a65-4a96-bb0d-8ae8f592e5c5 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:30:05.983139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367c72f-0a65-4a96-bb0d-8ae8f592e5c5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:30:05.983417 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367c72f-0a65-4a96-bb0d-8ae8f592e5c5 None None] PATH_INFO: `/v3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:30:05.984847 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 775/1517] 10.222.0.87 () {54 vars in 925 bytes} [Thu May 19 21:30:05 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:30:05.991874 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752179ac-3538-45c2-a233-2fc29d41a13a None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:30:05.992222 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752179ac-3538-45c2-a233-2fc29d41a13a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:30:05.992579 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752179ac-3538-45c2-a233-2fc29d41a13a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:30:06.017503 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-752179ac-3538-45c2-a233-2fc29d41a13a None None] MFA Rules not processed for user `f6143bddc05a46ea972709576c7ad95f`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:30:06.018669 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-752179ac-3538-45c2-a233-2fc29d41a13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:30:06.088319 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 743/1518] 10.222.0.87 () {58 vars in 1015 bytes} [Thu May 19 21:30:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:30:10.026207 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44757875-12e8-468a-aaf4-44e77f0d030c None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:30:10.026791 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44757875-12e8-468a-aaf4-44e77f0d030c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:30:10.026791 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44757875-12e8-468a-aaf4-44e77f0d030c None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:30:10.028230 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 776/1519] 10.222.0.65 () {54 vars in 960 bytes} [Thu May 19 21:30:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:30:10.033559 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf14e55-b45b-4f68-b65c-87a0605740a5 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:30:10.033982 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf14e55-b45b-4f68-b65c-87a0605740a5 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:30:10.034073 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf14e55-b45b-4f68-b65c-87a0605740a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:30:10.053073 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-8bf14e55-b45b-4f68-b65c-87a0605740a5 None None] MFA Rules not processed for user `f6143bddc05a46ea972709576c7ad95f`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:30:10.054307 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8bf14e55-b45b-4f68-b65c-87a0605740a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:30:10.063915 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 744/1520] 10.222.0.65 () {58 vars in 1057 bytes} [Thu May 19 21:30:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:30:10.074187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44757875-12e8-468a-aaf4-44e77f0d030c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:30:10.090742 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44757875-12e8-468a-aaf4-44e77f0d030c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:30:10.097624 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df04b69-3859-4b8c-82e4-75db873cd785 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6143bddc05a46ea972709576c7ad95f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:30:10.098678 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df04b69-3859-4b8c-82e4-75db873cd785 service placement] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:30:10.098963 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df04b69-3859-4b8c-82e4-75db873cd785 service placement] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:30:10.099266 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df04b69-3859-4b8c-82e4-75db873cd785 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:30:10.110253 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df04b69-3859-4b8c-82e4-75db873cd785 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:30:10.111861 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df04b69-3859-4b8c-82e4-75db873cd785 service placement] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:30:10.122532 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 777/1521] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:30:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:30:17.048940 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dadeb8-b64b-4933-8aa1-afa2452c53a6 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:30:17.049496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dadeb8-b64b-4933-8aa1-afa2452c53a6 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:30:17.049496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dadeb8-b64b-4933-8aa1-afa2452c53a6 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:30:17.050989 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 745/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 21:30:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:30:17.057447 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fef229-1ae7-41b0-917f-e362a939088a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:30:17.057982 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fef229-1ae7-41b0-917f-e362a939088a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:30:17.057982 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fef229-1ae7-41b0-917f-e362a939088a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:30:17.076597 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d6fef229-1ae7-41b0-917f-e362a939088a None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:30:17.077801 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d6fef229-1ae7-41b0-917f-e362a939088a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:30:17.085888 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 778/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 21:30:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:30:17.266033 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dadeb8-b64b-4933-8aa1-afa2452c53a6 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:30:17.283108 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dadeb8-b64b-4933-8aa1-afa2452c53a6 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:30:17.290465 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f3824-3577-4f36-99ce-1888c94fe7da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:30:17.291127 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f3824-3577-4f36-99ce-1888c94fe7da service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:30:17.291410 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f3824-3577-4f36-99ce-1888c94fe7da service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:30:17.291687 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f3824-3577-4f36-99ce-1888c94fe7da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:30:17.301611 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292f3824-3577-4f36-99ce-1888c94fe7da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:30:17.303253 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292f3824-3577-4f36-99ce-1888c94fe7da service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:30:17.312988 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 746/1524] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:30:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:30:18.322517 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeaf44f-9acb-4fa3-a62f-139b1379848c None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:30:18.323081 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeaf44f-9acb-4fa3-a62f-139b1379848c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:30:18.323081 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeaf44f-9acb-4fa3-a62f-139b1379848c None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:30:18.324438 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 779/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 19 21:30:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:30:18.331317 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299f7a1-b5ff-4635-92a7-51c2f7c94167 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:30:18.331906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299f7a1-b5ff-4635-92a7-51c2f7c94167 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:30:18.332299 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299f7a1-b5ff-4635-92a7-51c2f7c94167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:30:18.360874 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b299f7a1-b5ff-4635-92a7-51c2f7c94167 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:30:18.362615 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b299f7a1-b5ff-4635-92a7-51c2f7c94167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:30:18.374614 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 747/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 19 21:30:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:30:18.542446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeaf44f-9acb-4fa3-a62f-139b1379848c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:30:18.564070 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeaf44f-9acb-4fa3-a62f-139b1379848c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:30:18.570266 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab84d0dd-67db-4ae9-ab19-12b9064f1ceb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:30:18.570955 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab84d0dd-67db-4ae9-ab19-12b9064f1ceb service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:30:18.571242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab84d0dd-67db-4ae9-ab19-12b9064f1ceb service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:30:18.571524 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab84d0dd-67db-4ae9-ab19-12b9064f1ceb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:30:18.580280 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab84d0dd-67db-4ae9-ab19-12b9064f1ceb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:30:18.581830 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab84d0dd-67db-4ae9-ab19-12b9064f1ceb service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:30:18.592444 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 780/1527] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:30:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:30:23.071679 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ac395-e468-4857-960e-a3ac31cd7163 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:30:23.072260 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ac395-e468-4857-960e-a3ac31cd7163 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:30:23.072260 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ac395-e468-4857-960e-a3ac31cd7163 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:30:23.073619 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 748/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 21:30:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:30:23.079655 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f6528-b113-4dfe-92d4-59d52e25f25c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:30:23.080223 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f6528-b113-4dfe-92d4-59d52e25f25c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:30:23.080223 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f6528-b113-4dfe-92d4-59d52e25f25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:30:23.099555 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-eb3f6528-b113-4dfe-92d4-59d52e25f25c None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:30:23.101020 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-eb3f6528-b113-4dfe-92d4-59d52e25f25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:30:23.118862 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 781/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 21:30:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:30:23.159635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ac395-e468-4857-960e-a3ac31cd7163 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:30:23.161280 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-286ac395-e468-4857-960e-a3ac31cd7163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:30:23.228330 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ac395-e468-4857-960e-a3ac31cd7163 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:30:23.232501 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc43aaa7-f1bb-4670-9dd5-96d34d0450d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:30:23.233136 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43aaa7-f1bb-4670-9dd5-96d34d0450d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:30:23.233436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43aaa7-f1bb-4670-9dd5-96d34d0450d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:30:23.233706 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43aaa7-f1bb-4670-9dd5-96d34d0450d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:30:23.244367 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc43aaa7-f1bb-4670-9dd5-96d34d0450d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:30:23.245793 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc43aaa7-f1bb-4670-9dd5-96d34d0450d2 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:30:23.250934 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:30:23.256831 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 749/1530] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:30:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:30:25.675091 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3dbcfa-25de-438e-adca-94fc8d5ffe38 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:30:25.675659 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3dbcfa-25de-438e-adca-94fc8d5ffe38 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:30:25.675758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3dbcfa-25de-438e-adca-94fc8d5ffe38 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:30:25.677541 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 782/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 21:30:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:30:25.683731 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02724e-bab9-4a92-81fd-151268091919 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:30:25.684299 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02724e-bab9-4a92-81fd-151268091919 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:30:25.684299 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02724e-bab9-4a92-81fd-151268091919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:30:25.704219 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-4c02724e-bab9-4a92-81fd-151268091919 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:30:25.706218 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4c02724e-bab9-4a92-81fd-151268091919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:30:25.716228 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 750/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 21:30:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:30:25.757771 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3dbcfa-25de-438e-adca-94fc8d5ffe38 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:30:25.774099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3dbcfa-25de-438e-adca-94fc8d5ffe38 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:30:25.779806 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ed51a-ecd0-4fd1-995c-2abb8165916f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:30:25.780460 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ed51a-ecd0-4fd1-995c-2abb8165916f service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:30:25.780741 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ed51a-ecd0-4fd1-995c-2abb8165916f service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:30:25.781013 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ed51a-ecd0-4fd1-995c-2abb8165916f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:30:25.791955 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ed51a-ecd0-4fd1-995c-2abb8165916f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:30:25.793446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ed51a-ecd0-4fd1-995c-2abb8165916f service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:30:25.803407 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 783/1533] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:30:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:30:29.744107 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78666113-92a2-46cd-b00e-65b5ceeb1ca3 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:30:29.745007 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78666113-92a2-46cd-b00e-65b5ceeb1ca3 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:30:29.745007 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78666113-92a2-46cd-b00e-65b5ceeb1ca3 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:30:29.746365 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 751/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 21:30:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:30:29.752243 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ebb275-8128-4cfb-8dfb-84f94390d083 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:30:29.752767 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ebb275-8128-4cfb-8dfb-84f94390d083 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:30:29.752912 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ebb275-8128-4cfb-8dfb-84f94390d083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:30:29.774166 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f3ebb275-8128-4cfb-8dfb-84f94390d083 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:30:29.775426 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f3ebb275-8128-4cfb-8dfb-84f94390d083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:30:29.784906 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 784/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 21:30:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:30:30.065167 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78666113-92a2-46cd-b00e-65b5ceeb1ca3 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:30:30.067273 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-78666113-92a2-46cd-b00e-65b5ceeb1ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:30:30.132125 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78666113-92a2-46cd-b00e-65b5ceeb1ca3 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:30:30.136655 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a635f0-cb16-4332-9256-e16264f6ba8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:30:30.137418 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a635f0-cb16-4332-9256-e16264f6ba8a service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:30:30.137731 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a635f0-cb16-4332-9256-e16264f6ba8a service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:30:30.138012 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a635f0-cb16-4332-9256-e16264f6ba8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:30:30.150011 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a635f0-cb16-4332-9256-e16264f6ba8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:30:30.151521 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a635f0-cb16-4332-9256-e16264f6ba8a service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:30:30.161771 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 752/1536] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:30:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:30:32.467299 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0609f0-1e2b-4a49-a411-6600478cdb37 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:30:32.467881 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0609f0-1e2b-4a49-a411-6600478cdb37 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:30:32.467881 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0609f0-1e2b-4a49-a411-6600478cdb37 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:30:32.469273 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 785/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 19 21:30:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:30:32.475824 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0471bd-e4e0-45be-b2a0-7dfa26ca3b07 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:30:32.476370 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0471bd-e4e0-45be-b2a0-7dfa26ca3b07 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:30:32.476370 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0471bd-e4e0-45be-b2a0-7dfa26ca3b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:30:32.499076 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-bb0471bd-e4e0-45be-b2a0-7dfa26ca3b07 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:30:32.500284 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-bb0471bd-e4e0-45be-b2a0-7dfa26ca3b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:30:32.510024 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 753/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 19 21:30:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:30:32.784086 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0609f0-1e2b-4a49-a411-6600478cdb37 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:30:32.790340 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ab0609f0-1e2b-4a49-a411-6600478cdb37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:30:32.801567 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=65 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:30:32.807793 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0609f0-1e2b-4a49-a411-6600478cdb37 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:30:32.813989 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cef0d2-7ede-48ce-84c9-26c494a043cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:30:32.814951 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cef0d2-7ede-48ce-84c9-26c494a043cc service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:30:32.815231 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cef0d2-7ede-48ce-84c9-26c494a043cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:30:32.815505 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cef0d2-7ede-48ce-84c9-26c494a043cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:30:32.827833 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cef0d2-7ede-48ce-84c9-26c494a043cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:30:32.829386 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cef0d2-7ede-48ce-84c9-26c494a043cc service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:30:32.841821 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 786/1539] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:30:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:30:42.492982 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=32 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:30:42.836135 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:04.509047 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3571f-c2af-4f4d-adea-3cd0215e3b3a None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:04.510207 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3571f-c2af-4f4d-adea-3cd0215e3b3a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:04.510744 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3571f-c2af-4f4d-adea-3cd0215e3b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:04.538724 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-02f3571f-c2af-4f4d-adea-3cd0215e3b3a None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:04.539946 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-02f3571f-c2af-4f4d-adea-3cd0215e3b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:04.551240 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 754/1540] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:04.556152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ed37d-be87-4b32-bbc1-ba97962eeb4d None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:04.556738 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ed37d-be87-4b32-bbc1-ba97962eeb4d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:04.556738 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ed37d-be87-4b32-bbc1-ba97962eeb4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:04.562278 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ea5b8-57c7-4c3f-9165-2a924319340e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:04.562589 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ea5b8-57c7-4c3f-9165-2a924319340e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:04.562853 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ea5b8-57c7-4c3f-9165-2a924319340e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:04.582621 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-641ed37d-be87-4b32-bbc1-ba97962eeb4d None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:04.583854 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-641ed37d-be87-4b32-bbc1-ba97962eeb4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:04.590714 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-138ea5b8-57c7-4c3f-9165-2a924319340e None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:04.591940 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-138ea5b8-57c7-4c3f-9165-2a924319340e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:04.605246 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 755/1541] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:32:04.608280 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 787/1542] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:04.610339 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ea5b8-57c7-4c3f-9165-2a924319340e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:04.622996 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a06200-769b-4a5e-b8ed-d11aa1e2d136 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:04.623286 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a06200-769b-4a5e-b8ed-d11aa1e2d136 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:04.623559 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a06200-769b-4a5e-b8ed-d11aa1e2d136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:04.623678 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ea5b8-57c7-4c3f-9165-2a924319340e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:04.633605 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d2df8f-b5c8-4bd9-a389-0095f5cff210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:04.634334 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2df8f-b5c8-4bd9-a389-0095f5cff210 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:04.634625 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2df8f-b5c8-4bd9-a389-0095f5cff210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:04.634905 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2df8f-b5c8-4bd9-a389-0095f5cff210 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:04.635782 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d2df8f-b5c8-4bd9-a389-0095f5cff210 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:32:04.636106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d2df8f-b5c8-4bd9-a389-0095f5cff210 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:04.637903 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d2df8f-b5c8-4bd9-a389-0095f5cff210 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:04.662718 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 756/1543] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:32:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:32:04.671802 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d2df8f-b5c8-4bd9-a389-0095f5cff210 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:04.680687 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-04a06200-769b-4a5e-b8ed-d11aa1e2d136 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:04.681936 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-04a06200-769b-4a5e-b8ed-d11aa1e2d136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:04.698074 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d2df8f-b5c8-4bd9-a389-0095f5cff210 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:04.706377 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e5389-3827-42d1-9b89-d2803fcbee9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:04.707152 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e5389-3827-42d1-9b89-d2803fcbee9f admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:04.707759 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e5389-3827-42d1-9b89-d2803fcbee9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:04.708050 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e5389-3827-42d1-9b89-d2803fcbee9f admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:04.708774 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 788/1544] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:04.708931 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96e5389-3827-42d1-9b89-d2803fcbee9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:04.709859 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: WARNING py.warnings [None req-c96e5389-3827-42d1-9b89-d2803fcbee9f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 21:32:04.709859 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: warnings.warn(msg) May 19 21:32:04.709859 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: May 19 21:32:04.714034 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96e5389-3827-42d1-9b89-d2803fcbee9f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:04.714456 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a06200-769b-4a5e-b8ed-d11aa1e2d136 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:04.729420 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a06200-769b-4a5e-b8ed-d11aa1e2d136 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:04.745801 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78db454-d188-47d3-9bdf-8a9916f45feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:04.746524 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78db454-d188-47d3-9bdf-8a9916f45feb admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:04.746808 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78db454-d188-47d3-9bdf-8a9916f45feb admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:04.747089 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78db454-d188-47d3-9bdf-8a9916f45feb admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:04.748067 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78db454-d188-47d3-9bdf-8a9916f45feb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:32:04.748379 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78db454-d188-47d3-9bdf-8a9916f45feb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:04.751099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: WARNING py.warnings [None req-b78db454-d188-47d3-9bdf-8a9916f45feb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 19 21:32:04.751099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: warnings.warn(msg) May 19 21:32:04.751099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: May 19 21:32:04.752199 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78db454-d188-47d3-9bdf-8a9916f45feb admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:04.769988 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 757/1545] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:32:04 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:32:04.772920 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 789/1546] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:32:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:32:04.774767 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e5389-3827-42d1-9b89-d2803fcbee9f admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:04.781637 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78db454-d188-47d3-9bdf-8a9916f45feb admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:04.807708 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78db454-d188-47d3-9bdf-8a9916f45feb admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:04.927299 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e5389-3827-42d1-9b89-d2803fcbee9f admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:04.947282 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64004adf-afe3-4504-946b-ec1bed04561c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:04.949000 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64004adf-afe3-4504-946b-ec1bed04561c admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:04.950037 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64004adf-afe3-4504-946b-ec1bed04561c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:04.951712 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64004adf-afe3-4504-946b-ec1bed04561c admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:04.953277 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64004adf-afe3-4504-946b-ec1bed04561c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:04.957168 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa512549-4ae2-4474-b38a-b90ca6c19a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:04.963302 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64004adf-afe3-4504-946b-ec1bed04561c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:04.963922 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa512549-4ae2-4474-b38a-b90ca6c19a58 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:04.964525 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa512549-4ae2-4474-b38a-b90ca6c19a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:04.965898 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa512549-4ae2-4474-b38a-b90ca6c19a58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:04.967273 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa512549-4ae2-4474-b38a-b90ca6c19a58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:04.974292 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa512549-4ae2-4474-b38a-b90ca6c19a58 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:05.017902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 758/1547] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:32:04 2022] POST /identity/v3/users => generated 290 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:32:05.019841 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43e751-8b04-4d5f-a472-1e431ee422ff None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:05.021683 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43e751-8b04-4d5f-a472-1e431ee422ff None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:05.022193 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43e751-8b04-4d5f-a472-1e431ee422ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:05.045653 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 790/1548] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:32:04 2022] POST /identity/v3/projects => generated 405 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:32:05.047381 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3ec6e-3e54-4306-bf57-fee8becfe7a3 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:05.047903 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3ec6e-3e54-4306-bf57-fee8becfe7a3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:05.048386 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3ec6e-3e54-4306-bf57-fee8becfe7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:05.070268 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-0b43e751-8b04-4d5f-a472-1e431ee422ff None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:05.076098 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0b43e751-8b04-4d5f-a472-1e431ee422ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:05.088784 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-2fb3ec6e-3e54-4306-bf57-fee8becfe7a3 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:05.091559 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2fb3ec6e-3e54-4306-bf57-fee8becfe7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:05.163565 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 759/1549] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:05.164753 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b43e751-8b04-4d5f-a472-1e431ee422ff None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:05.174260 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b43e751-8b04-4d5f-a472-1e431ee422ff None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:05.175193 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 791/1550] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:05.176777 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387aedb-236b-4fd1-83d2-b3306791c58d None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:05.177065 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387aedb-236b-4fd1-83d2-b3306791c58d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:05.177347 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387aedb-236b-4fd1-83d2-b3306791c58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:05.186224 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682cf78-7d36-446a-a655-c5f505f225d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:05.186997 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682cf78-7d36-446a-a655-c5f505f225d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:05.187274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682cf78-7d36-446a-a655-c5f505f225d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:05.187544 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682cf78-7d36-446a-a655-c5f505f225d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:05.189224 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c682cf78-7d36-446a-a655-c5f505f225d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:05.190898 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c682cf78-7d36-446a-a655-c5f505f225d8 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:05.201304 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 760/1551] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:05.201527 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a387aedb-236b-4fd1-83d2-b3306791c58d None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:05.202060 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682cf78-7d36-446a-a655-c5f505f225d8 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:05.202523 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a387aedb-236b-4fd1-83d2-b3306791c58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:05.210933 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 792/1552] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:05.211928 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682cf78-7d36-446a-a655-c5f505f225d8 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:05.215599 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5410d653-c753-4593-837c-762488715455 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:05.215888 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5410d653-c753-4593-837c-762488715455 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:05.216164 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5410d653-c753-4593-837c-762488715455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:05.216680 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a3e99-c27d-493a-a6d5-e5480585c456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:05.217332 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a3e99-c27d-493a-a6d5-e5480585c456 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:05.217611 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a3e99-c27d-493a-a6d5-e5480585c456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:05.217886 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a3e99-c27d-493a-a6d5-e5480585c456 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:05.218611 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785a3e99-c27d-493a-a6d5-e5480585c456 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:05.220573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785a3e99-c27d-493a-a6d5-e5480585c456 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:05.235981 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-5410d653-c753-4593-837c-762488715455 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:05.237214 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5410d653-c753-4593-837c-762488715455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:05.245305 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 761/1553] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:32:05 2022] POST /identity/v3/users => generated 303 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:05.246397 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 793/1554] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:05.246635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81657e-5921-4d0b-9c33-7290188b676c None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:05.246927 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81657e-5921-4d0b-9c33-7290188b676c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:05.247205 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81657e-5921-4d0b-9c33-7290188b676c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:05.248627 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f437086-2859-4210-b062-312b2139544f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:05.249097 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f437086-2859-4210-b062-312b2139544f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:05.249541 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f437086-2859-4210-b062-312b2139544f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:05.269483 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-2c81657e-5921-4d0b-9c33-7290188b676c None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:05.271283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-2c81657e-5921-4d0b-9c33-7290188b676c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:05.274431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-2f437086-2859-4210-b062-312b2139544f None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:05.275549 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2f437086-2859-4210-b062-312b2139544f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:05.279643 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 762/1555] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:05.280906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15d0f2-59a8-450f-bdb7-a4eb2b4dac1d None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:05.281196 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15d0f2-59a8-450f-bdb7-a4eb2b4dac1d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:05.281478 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15d0f2-59a8-450f-bdb7-a4eb2b4dac1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:05.286035 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 794/1556] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:05.287432 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80861c92-f89d-4d93-8b88-c88a66925c98 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:05.287719 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80861c92-f89d-4d93-8b88-c88a66925c98 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:05.287994 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80861c92-f89d-4d93-8b88-c88a66925c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:05.301030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:05.307344 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-80861c92-f89d-4d93-8b88-c88a66925c98 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:05.308153 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-aa15d0f2-59a8-450f-bdb7-a4eb2b4dac1d None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:05.308657 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-80861c92-f89d-4d93-8b88-c88a66925c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:05.309847 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-aa15d0f2-59a8-450f-bdb7-a4eb2b4dac1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:05.316712 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 795/1557] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:05.318119 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80861c92-f89d-4d93-8b88-c88a66925c98 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:05.318285 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 763/1558] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:05.322646 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d22fb8-ca61-4b8b-a00a-55d0b92d8d31 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:05.322937 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d22fb8-ca61-4b8b-a00a-55d0b92d8d31 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:05.323214 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d22fb8-ca61-4b8b-a00a-55d0b92d8d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:05.327638 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80861c92-f89d-4d93-8b88-c88a66925c98 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:05.333979 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ecabf-f383-4f36-9d74-42e1ddc2b04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:05.334697 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ecabf-f383-4f36-9d74-42e1ddc2b04e admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:05.334983 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ecabf-f383-4f36-9d74-42e1ddc2b04e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:05.335262 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ecabf-f383-4f36-9d74-42e1ddc2b04e admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:05.336000 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931ecabf-f383-4f36-9d74-42e1ddc2b04e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:05.337625 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931ecabf-f383-4f36-9d74-42e1ddc2b04e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:05.344642 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b3d22fb8-ca61-4b8b-a00a-55d0b92d8d31 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:05.345852 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b3d22fb8-ca61-4b8b-a00a-55d0b92d8d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:05.348692 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 796/1559] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:05.349657 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ecabf-f383-4f36-9d74-42e1ddc2b04e admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:05.358630 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 764/1560] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:05.361634 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ecabf-f383-4f36-9d74-42e1ddc2b04e admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:05.363391 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a694bd-fae7-49ae-a61a-93e951a11088 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:05.363772 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a694bd-fae7-49ae-a61a-93e951a11088 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:05.364029 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a694bd-fae7-49ae-a61a-93e951a11088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:05.367432 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3710c8a1-c8ff-42ff-aaa0-4b27ffd8473b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:05.368119 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710c8a1-c8ff-42ff-aaa0-4b27ffd8473b admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:05.368399 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710c8a1-c8ff-42ff-aaa0-4b27ffd8473b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:05.368672 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710c8a1-c8ff-42ff-aaa0-4b27ffd8473b admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:05.369429 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3710c8a1-c8ff-42ff-aaa0-4b27ffd8473b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:05.371102 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3710c8a1-c8ff-42ff-aaa0-4b27ffd8473b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:05.381392 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 797/1561] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:05.381801 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-19a694bd-fae7-49ae-a61a-93e951a11088 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:05.382676 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df29465-35ba-4563-8ca1-498c6fae3f3e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:05.382967 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df29465-35ba-4563-8ca1-498c6fae3f3e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:05.383069 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-19a694bd-fae7-49ae-a61a-93e951a11088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:05.383247 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df29465-35ba-4563-8ca1-498c6fae3f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:05.391130 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 765/1562] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:05.392382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21569d28-3235-40f0-af4b-839c983bce35 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:05.392672 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21569d28-3235-40f0-af4b-839c983bce35 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:05.392951 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21569d28-3235-40f0-af4b-839c983bce35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:05.403109 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-8df29465-35ba-4563-8ca1-498c6fae3f3e None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:05.404370 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8df29465-35ba-4563-8ca1-498c6fae3f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:05.411455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-21569d28-3235-40f0-af4b-839c983bce35 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:05.412458 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-21569d28-3235-40f0-af4b-839c983bce35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:05.413042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 798/1563] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:05.413803 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df29465-35ba-4563-8ca1-498c6fae3f3e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:05.420035 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:05.421113 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 766/1564] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:05.421956 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21569d28-3235-40f0-af4b-839c983bce35 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:05.423644 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df29465-35ba-4563-8ca1-498c6fae3f3e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:05.430165 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b08aee3-6bbf-402b-b6b2-57d52f97f0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:05.430789 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08aee3-6bbf-402b-b6b2-57d52f97f0d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:05.431237 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08aee3-6bbf-402b-b6b2-57d52f97f0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:05.431509 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08aee3-6bbf-402b-b6b2-57d52f97f0d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:05.431964 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21569d28-3235-40f0-af4b-839c983bce35 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:05.432231 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b08aee3-6bbf-402b-b6b2-57d52f97f0d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:32:05.432538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b08aee3-6bbf-402b-b6b2-57d52f97f0d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:05.434151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b08aee3-6bbf-402b-b6b2-57d52f97f0d9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:05.436402 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9de11a-ded7-44a5-b63f-5882f889b6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:05.437052 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9de11a-ded7-44a5-b63f-5882f889b6b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:05.437474 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9de11a-ded7-44a5-b63f-5882f889b6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:05.437756 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9de11a-ded7-44a5-b63f-5882f889b6b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:05.438509 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9de11a-ded7-44a5-b63f-5882f889b6b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:32:05.438813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9de11a-ded7-44a5-b63f-5882f889b6b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:05.440398 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9de11a-ded7-44a5-b63f-5882f889b6b3 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:05.446916 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 799/1565] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:32:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:32:05.447698 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b08aee3-6bbf-402b-b6b2-57d52f97f0d9 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:05.457178 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b08aee3-6bbf-402b-b6b2-57d52f97f0d9 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:05.458623 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 767/1566] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:32:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:32:05.459649 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9de11a-ded7-44a5-b63f-5882f889b6b3 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:05.462767 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352d7ac3-ba47-452d-8754-6817e8dcaba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:05.463656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d7ac3-ba47-452d-8754-6817e8dcaba6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:05.463932 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d7ac3-ba47-452d-8754-6817e8dcaba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:05.464205 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d7ac3-ba47-452d-8754-6817e8dcaba6 admin admin] PATH_INFO: `/v3/projects/6166dc89ce0545c1bbb8adb947034e79/users/bc570204a2bb4889889a67850dd1d9fe/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:05.470783 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9de11a-ded7-44a5-b63f-5882f889b6b3 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:05.475926 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3074e0ef-b6db-44e1-bb68-aecaa39e3886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:05.476580 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074e0ef-b6db-44e1-bb68-aecaa39e3886 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:05.476859 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074e0ef-b6db-44e1-bb68-aecaa39e3886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:05.477133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074e0ef-b6db-44e1-bb68-aecaa39e3886 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:05.477860 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3074e0ef-b6db-44e1-bb68-aecaa39e3886 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:32:05.478164 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3074e0ef-b6db-44e1-bb68-aecaa39e3886 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:05.479796 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3074e0ef-b6db-44e1-bb68-aecaa39e3886 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:05.480352 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352d7ac3-ba47-452d-8754-6817e8dcaba6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6166dc89ce0545c1bbb8adb947034e79, user_id=bc570204a2bb4889889a67850dd1d9fe, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:05.483731 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352d7ac3-ba47-452d-8754-6817e8dcaba6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:05.494022 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 768/1567] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:32:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:32:05.495038 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3074e0ef-b6db-44e1-bb68-aecaa39e3886 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:05.501346 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 800/1568] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:05 2022] PUT /identity/v3/projects/6166dc89ce0545c1bbb8adb947034e79/users/bc570204a2bb4889889a67850dd1d9fe/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:05.502870 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352d7ac3-ba47-452d-8754-6817e8dcaba6 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:05.506544 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3074e0ef-b6db-44e1-bb68-aecaa39e3886 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:05.564776 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00850020-94f6-464e-ae15-6e60094f4917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:05.565493 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00850020-94f6-464e-ae15-6e60094f4917 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:05.565642 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00850020-94f6-464e-ae15-6e60094f4917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:05.565916 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00850020-94f6-464e-ae15-6e60094f4917 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:05.566973 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00850020-94f6-464e-ae15-6e60094f4917 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:05.568502 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352d7ac3-ba47-452d-8754-6817e8dcaba6 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:05.568950 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00850020-94f6-464e-ae15-6e60094f4917 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:05.573183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a0c4b-59a0-4581-b1fe-04fcaf3eb11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:05.573852 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a0c4b-59a0-4581-b1fe-04fcaf3eb11f admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:05.574151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a0c4b-59a0-4581-b1fe-04fcaf3eb11f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:05.574459 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a0c4b-59a0-4581-b1fe-04fcaf3eb11f admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:05.575192 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20a0c4b-59a0-4581-b1fe-04fcaf3eb11f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:32:05.575499 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20a0c4b-59a0-4581-b1fe-04fcaf3eb11f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:05.577097 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20a0c4b-59a0-4581-b1fe-04fcaf3eb11f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:05.578956 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 769/1569] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:05 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:05.579838 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00850020-94f6-464e-ae15-6e60094f4917 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:05.593969 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 801/1570] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:32:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:32:05.593969 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a0c4b-59a0-4581-b1fe-04fcaf3eb11f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:05.594288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00850020-94f6-464e-ae15-6e60094f4917 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:05.598521 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43672726-e1b1-4c48-bd8f-533bf32beb28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:05.599187 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43672726-e1b1-4c48-bd8f-533bf32beb28 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:05.599498 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43672726-e1b1-4c48-bd8f-533bf32beb28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:05.599775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43672726-e1b1-4c48-bd8f-533bf32beb28 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:05.600492 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43672726-e1b1-4c48-bd8f-533bf32beb28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:32:05.600794 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43672726-e1b1-4c48-bd8f-533bf32beb28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:05.601520 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a0c4b-59a0-4581-b1fe-04fcaf3eb11f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:05.602454 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43672726-e1b1-4c48-bd8f-533bf32beb28 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:05.606456 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13f362-2a7e-4f79-a394-d6fe21661210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:05.607109 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13f362-2a7e-4f79-a394-d6fe21661210 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:05.607519 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13f362-2a7e-4f79-a394-d6fe21661210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:05.607660 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13f362-2a7e-4f79-a394-d6fe21661210 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:05.608407 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc13f362-2a7e-4f79-a394-d6fe21661210 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:32:05.608715 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc13f362-2a7e-4f79-a394-d6fe21661210 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:05.610318 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc13f362-2a7e-4f79-a394-d6fe21661210 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:05.614886 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 770/1571] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:32:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:32:05.615731 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43672726-e1b1-4c48-bd8f-533bf32beb28 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:05.625415 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43672726-e1b1-4c48-bd8f-533bf32beb28 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:05.627741 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 802/1572] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:32:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:32:05.628821 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13f362-2a7e-4f79-a394-d6fe21661210 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:05.637116 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e1e2ac-997a-42ad-954b-fac258db713c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:05.637787 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e1e2ac-997a-42ad-954b-fac258db713c admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:05.638065 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e1e2ac-997a-42ad-954b-fac258db713c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:05.638369 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e1e2ac-997a-42ad-954b-fac258db713c admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:05.639058 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e1e2ac-997a-42ad-954b-fac258db713c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:05.640982 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e1e2ac-997a-42ad-954b-fac258db713c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:05.644566 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13f362-2a7e-4f79-a394-d6fe21661210 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:05.649909 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cae9f6-5915-4942-bc0e-5de4dbe2c5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:05.650617 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cae9f6-5915-4942-bc0e-5de4dbe2c5ea admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:05.650900 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cae9f6-5915-4942-bc0e-5de4dbe2c5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:05.651183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cae9f6-5915-4942-bc0e-5de4dbe2c5ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:05.651882 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cae9f6-5915-4942-bc0e-5de4dbe2c5ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:05.653780 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cae9f6-5915-4942-bc0e-5de4dbe2c5ea admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:05.670539 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 771/1573] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:32:05 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:05.670539 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e1e2ac-997a-42ad-954b-fac258db713c admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:05.688359 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 803/1574] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:32:05 2022] POST /identity/v3/projects => generated 369 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:05.688359 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498c09c-433d-444b-9e4c-1a87716e5d5f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:05.688359 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498c09c-433d-444b-9e4c-1a87716e5d5f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:05.688359 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498c09c-433d-444b-9e4c-1a87716e5d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:05.706868 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d498c09c-433d-444b-9e4c-1a87716e5d5f None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:05.708200 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d498c09c-433d-444b-9e4c-1a87716e5d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:05.736451 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e1e2ac-997a-42ad-954b-fac258db713c admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:05.755689 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:05.772720 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 804/1575] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:05.773868 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498c09c-433d-444b-9e4c-1a87716e5d5f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:05.785961 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498c09c-433d-444b-9e4c-1a87716e5d5f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:05.791881 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ab868-93a5-4dbe-ab38-1ddcf29ba487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:05.792795 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ab868-93a5-4dbe-ab38-1ddcf29ba487 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:05.793082 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ab868-93a5-4dbe-ab38-1ddcf29ba487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:05.793361 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ab868-93a5-4dbe-ab38-1ddcf29ba487 admin admin] PATH_INFO: `/v3/projects/c6babff9bc984c8f9ad6ffa24ea4b507/users/aa22d9b1a5734acc889ccda089459177/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:05.797006 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9642264-ab91-4510-ae43-9854614525eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:05.797681 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9642264-ab91-4510-ae43-9854614525eb admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:05.797967 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9642264-ab91-4510-ae43-9854614525eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:05.798268 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9642264-ab91-4510-ae43-9854614525eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:05.798987 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9642264-ab91-4510-ae43-9854614525eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:05.801023 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9642264-ab91-4510-ae43-9854614525eb admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:05.809879 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428ab868-93a5-4dbe-ab38-1ddcf29ba487 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6babff9bc984c8f9ad6ffa24ea4b507, user_id=aa22d9b1a5734acc889ccda089459177, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:05.812719 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428ab868-93a5-4dbe-ab38-1ddcf29ba487 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:05.829467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 805/1576] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:05 2022] PUT /identity/v3/projects/c6babff9bc984c8f9ad6ffa24ea4b507/users/aa22d9b1a5734acc889ccda089459177/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:05.829467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ab868-93a5-4dbe-ab38-1ddcf29ba487 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:05.833977 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:05.839390 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 772/1577] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:32:05 2022] POST /identity/v3/projects => generated 391 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:05.839390 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9642264-ab91-4510-ae43-9854614525eb admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:05.904187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ab868-93a5-4dbe-ab38-1ddcf29ba487 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:05.904573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9642264-ab91-4510-ae43-9854614525eb admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:05.909406 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd4c5a-969d-4019-bd1f-3404739e8afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:05.910042 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c256b083-1062-4582-a815-22b61f70d32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:05.910188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd4c5a-969d-4019-bd1f-3404739e8afd admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:05.910387 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd4c5a-969d-4019-bd1f-3404739e8afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:05.910705 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd4c5a-969d-4019-bd1f-3404739e8afd admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:05.910928 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256b083-1062-4582-a815-22b61f70d32d admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:05.911243 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256b083-1062-4582-a815-22b61f70d32d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:05.911483 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd4c5a-969d-4019-bd1f-3404739e8afd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:05.911603 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256b083-1062-4582-a815-22b61f70d32d admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:05.912271 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c256b083-1062-4582-a815-22b61f70d32d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:05.913426 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd4c5a-969d-4019-bd1f-3404739e8afd admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:05.914180 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c256b083-1062-4582-a815-22b61f70d32d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:05.940673 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 806/1578] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:32:05 2022] POST /identity/v3/projects => generated 389 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:05.941841 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd4c5a-969d-4019-bd1f-3404739e8afd admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:05.942411 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 773/1579] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:32:05 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:05.943310 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c256b083-1062-4582-a815-22b61f70d32d admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:06.014287 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd4c5a-969d-4019-bd1f-3404739e8afd admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:06.018974 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69f8ed8-a560-4f1e-9f55-19df9e67edcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:06.019653 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69f8ed8-a560-4f1e-9f55-19df9e67edcb admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:06.019987 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69f8ed8-a560-4f1e-9f55-19df9e67edcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:06.020305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69f8ed8-a560-4f1e-9f55-19df9e67edcb admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:06.021028 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69f8ed8-a560-4f1e-9f55-19df9e67edcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:06.023001 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69f8ed8-a560-4f1e-9f55-19df9e67edcb admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:06.023133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c256b083-1062-4582-a815-22b61f70d32d admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:06.027898 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0de68-e28a-4b68-b76e-e246a7e71cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:06.028570 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0de68-e28a-4b68-b76e-e246a7e71cad admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:06.028850 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0de68-e28a-4b68-b76e-e246a7e71cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:06.029128 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0de68-e28a-4b68-b76e-e246a7e71cad admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:06.029817 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e0de68-e28a-4b68-b76e-e246a7e71cad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:06.031779 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e0de68-e28a-4b68-b76e-e246a7e71cad admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:06.051169 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 807/1580] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:32:05 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:06.052097 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69f8ed8-a560-4f1e-9f55-19df9e67edcb admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:06.056472 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 774/1581] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:32:05 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:06.059687 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0de68-e28a-4b68-b76e-e246a7e71cad admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:06.087065 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:06.118658 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69f8ed8-a560-4f1e-9f55-19df9e67edcb admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:06.123876 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5fcbd4-2f8c-41bb-921d-0d36f5c2374c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:06.124574 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5fcbd4-2f8c-41bb-921d-0d36f5c2374c admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:06.124843 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5fcbd4-2f8c-41bb-921d-0d36f5c2374c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:06.125121 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5fcbd4-2f8c-41bb-921d-0d36f5c2374c admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:06.125833 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5fcbd4-2f8c-41bb-921d-0d36f5c2374c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:06.126888 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: WARNING py.warnings [None req-ee5fcbd4-2f8c-41bb-921d-0d36f5c2374c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 21:32:06.126888 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: warnings.warn(msg) May 19 21:32:06.126888 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: May 19 21:32:06.128090 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5fcbd4-2f8c-41bb-921d-0d36f5c2374c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:06.128199 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0de68-e28a-4b68-b76e-e246a7e71cad admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:06.149532 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 808/1582] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:32:06 2022] POST /identity/v3/users => generated 285 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:06.151007 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d34a64-a53b-4214-8db1-de99977174cc None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:06.151292 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d34a64-a53b-4214-8db1-de99977174cc None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:06.151571 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d34a64-a53b-4214-8db1-de99977174cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:06.158473 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:06.184261 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c5d34a64-a53b-4214-8db1-de99977174cc None None] MFA Rules not processed for user `aa22d9b1a5734acc889ccda089459177`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:06.185615 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c5d34a64-a53b-4214-8db1-de99977174cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:06.189219 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244c78d4-c0cb-48da-a9a6-18e8d86b504f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:06.189884 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c78d4-c0cb-48da-a9a6-18e8d86b504f service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:06.190164 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c78d4-c0cb-48da-a9a6-18e8d86b504f service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:06.190464 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c78d4-c0cb-48da-a9a6-18e8d86b504f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:06.200163 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244c78d4-c0cb-48da-a9a6-18e8d86b504f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:06.201609 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244c78d4-c0cb-48da-a9a6-18e8d86b504f service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:06.211549 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 775/1583] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:06.212410 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244c78d4-c0cb-48da-a9a6-18e8d86b504f service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:06.222220 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244c78d4-c0cb-48da-a9a6-18e8d86b504f service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:06.227329 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b720ac-57c9-4248-ad9a-c5e24de61b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:06.227997 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b720ac-57c9-4248-ad9a-c5e24de61b76 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:06.228297 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b720ac-57c9-4248-ad9a-c5e24de61b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:06.228576 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b720ac-57c9-4248-ad9a-c5e24de61b76 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:06.229263 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b720ac-57c9-4248-ad9a-c5e24de61b76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:06.231224 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b720ac-57c9-4248-ad9a-c5e24de61b76 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:06.265848 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 776/1584] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:32:06 2022] POST /identity/v3/users => generated 296 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:06.265848 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b720ac-57c9-4248-ad9a-c5e24de61b76 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:06.278497 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b720ac-57c9-4248-ad9a-c5e24de61b76 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:06.278819 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 809/1585] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:06 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:06.279848 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d34a64-a53b-4214-8db1-de99977174cc None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:06.285219 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a694929-3b01-45b9-bbf1-7476445f8ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:06.286424 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a694929-3b01-45b9-bbf1-7476445f8ccc admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:06.286835 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a694929-3b01-45b9-bbf1-7476445f8ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:06.287220 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a694929-3b01-45b9-bbf1-7476445f8ccc admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:06.288147 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a694929-3b01-45b9-bbf1-7476445f8ccc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:06.291108 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a694929-3b01-45b9-bbf1-7476445f8ccc admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:06.293498 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d34a64-a53b-4214-8db1-de99977174cc None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:06.299150 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73d54e0-8e15-465e-ba89-87ccce3a2996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:06.300141 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d54e0-8e15-465e-ba89-87ccce3a2996 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:06.300555 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d54e0-8e15-465e-ba89-87ccce3a2996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:06.300943 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d54e0-8e15-465e-ba89-87ccce3a2996 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:06.301714 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73d54e0-8e15-465e-ba89-87ccce3a2996 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:06.304061 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73d54e0-8e15-465e-ba89-87ccce3a2996 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:06.330485 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 810/1587] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:32:06 2022] POST /identity/v3/users => generated 289 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:06.330807 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73d54e0-8e15-465e-ba89-87ccce3a2996 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:06.331032 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 777/1587] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:32:06 2022] POST /identity/v3/users => generated 295 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:06.333003 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a694929-3b01-45b9-bbf1-7476445f8ccc admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:06.341005 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73d54e0-8e15-465e-ba89-87ccce3a2996 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:06.346644 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f81aeba-c676-4e98-912f-9e9009f4ec7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:06.347770 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f81aeba-c676-4e98-912f-9e9009f4ec7d admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:06.348409 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f81aeba-c676-4e98-912f-9e9009f4ec7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:06.348664 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f81aeba-c676-4e98-912f-9e9009f4ec7d admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:06.349893 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f81aeba-c676-4e98-912f-9e9009f4ec7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:06.351379 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a694929-3b01-45b9-bbf1-7476445f8ccc admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:06.352324 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f81aeba-c676-4e98-912f-9e9009f4ec7d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:06.357562 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e1437f-eedd-4e45-be1d-ae2835f8935d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:06.358508 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e1437f-eedd-4e45-be1d-ae2835f8935d admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:06.358709 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e1437f-eedd-4e45-be1d-ae2835f8935d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:06.358989 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e1437f-eedd-4e45-be1d-ae2835f8935d admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:06.359741 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e1437f-eedd-4e45-be1d-ae2835f8935d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:06.361606 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e1437f-eedd-4e45-be1d-ae2835f8935d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:06.373576 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 778/1588] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:06.376098 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 811/1589] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:32:06 2022] POST /identity/v3/users => generated 287 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:06.376305 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e1437f-eedd-4e45-be1d-ae2835f8935d admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:06.376854 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f81aeba-c676-4e98-912f-9e9009f4ec7d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:06.391299 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f81aeba-c676-4e98-912f-9e9009f4ec7d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:06.395549 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e1437f-eedd-4e45-be1d-ae2835f8935d admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:06.397441 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7a92ff-5593-4a47-819c-42401d554bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:06.398268 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a92ff-5593-4a47-819c-42401d554bbd admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:06.398675 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a92ff-5593-4a47-819c-42401d554bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:06.399043 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a92ff-5593-4a47-819c-42401d554bbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:06.399836 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7a92ff-5593-4a47-819c-42401d554bbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:06.402764 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7a92ff-5593-4a47-819c-42401d554bbd admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:06.403979 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d4a375-a647-404f-b69c-55e2c92faaac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:06.405245 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4a375-a647-404f-b69c-55e2c92faaac admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:06.405741 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4a375-a647-404f-b69c-55e2c92faaac admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:06.406251 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4a375-a647-404f-b69c-55e2c92faaac admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:06.407628 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d4a375-a647-404f-b69c-55e2c92faaac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:06.410816 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d4a375-a647-404f-b69c-55e2c92faaac admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:06.419055 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 812/1590] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:06.420241 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7a92ff-5593-4a47-819c-42401d554bbd admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:06.433396 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 779/1591] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:06.435274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d4a375-a647-404f-b69c-55e2c92faaac admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:06.435944 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7a92ff-5593-4a47-819c-42401d554bbd admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:06.443653 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35312cb-d7da-454d-a873-afd3291029c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:06.444888 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35312cb-d7da-454d-a873-afd3291029c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:06.445406 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35312cb-d7da-454d-a873-afd3291029c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:06.446412 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35312cb-d7da-454d-a873-afd3291029c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:06.453544 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d4a375-a647-404f-b69c-55e2c92faaac admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:06.455902 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35312cb-d7da-454d-a873-afd3291029c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:06.462487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35312cb-d7da-454d-a873-afd3291029c8 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:06.462638 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83092c4-4837-49fc-8cb2-444c63f13f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:06.462638 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83092c4-4837-49fc-8cb2-444c63f13f3d admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:06.462992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83092c4-4837-49fc-8cb2-444c63f13f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:06.463420 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83092c4-4837-49fc-8cb2-444c63f13f3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:06.464760 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83092c4-4837-49fc-8cb2-444c63f13f3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:06.468285 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83092c4-4837-49fc-8cb2-444c63f13f3d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:06.488597 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 780/1592] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:06.491283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83092c4-4837-49fc-8cb2-444c63f13f3d admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:06.505431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 813/1593] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:32:06 2022] POST /identity/v3/projects => generated 405 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:06.506340 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35312cb-d7da-454d-a873-afd3291029c8 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:06.507508 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83092c4-4837-49fc-8cb2-444c63f13f3d admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:06.586446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35312cb-d7da-454d-a873-afd3291029c8 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:06.592197 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b32e1a-33c4-45c8-a637-25d7ce9a8b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:06.592912 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b32e1a-33c4-45c8-a637-25d7ce9a8b25 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:06.593271 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b32e1a-33c4-45c8-a637-25d7ce9a8b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:06.593621 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b32e1a-33c4-45c8-a637-25d7ce9a8b25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:06.593953 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2805882-ec61-44f0-884b-d1f2691a70a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:06.595224 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b32e1a-33c4-45c8-a637-25d7ce9a8b25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:06.595351 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2805882-ec61-44f0-884b-d1f2691a70a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:06.595351 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2805882-ec61-44f0-884b-d1f2691a70a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:06.595773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2805882-ec61-44f0-884b-d1f2691a70a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:06.596554 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b32e1a-33c4-45c8-a637-25d7ce9a8b25 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:06.596736 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2805882-ec61-44f0-884b-d1f2691a70a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:06.599077 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2805882-ec61-44f0-884b-d1f2691a70a6 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:06.612595 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 814/1594] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:06.612595 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b32e1a-33c4-45c8-a637-25d7ce9a8b25 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:06.612895 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 781/1595] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:06.612895 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2805882-ec61-44f0-884b-d1f2691a70a6 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:06.625758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b32e1a-33c4-45c8-a637-25d7ce9a8b25 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:06.632054 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38c2cf6-5ed3-420e-9611-df31b34e416f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:06.632840 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c2cf6-5ed3-420e-9611-df31b34e416f admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:06.633126 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c2cf6-5ed3-420e-9611-df31b34e416f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:06.633402 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c2cf6-5ed3-420e-9611-df31b34e416f admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:06.634218 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38c2cf6-5ed3-420e-9611-df31b34e416f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:06.636015 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38c2cf6-5ed3-420e-9611-df31b34e416f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:06.648943 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 815/1596] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:06.650000 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38c2cf6-5ed3-420e-9611-df31b34e416f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:06.653700 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2805882-ec61-44f0-884b-d1f2691a70a6 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:06.661259 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c2d5d0-5fe0-4f2e-9518-3138a659b16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:06.662264 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38c2cf6-5ed3-420e-9611-df31b34e416f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:06.662491 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2d5d0-5fe0-4f2e-9518-3138a659b16c admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:06.662773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2d5d0-5fe0-4f2e-9518-3138a659b16c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:06.663168 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2d5d0-5fe0-4f2e-9518-3138a659b16c admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:06.664718 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c2d5d0-5fe0-4f2e-9518-3138a659b16c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:06.667841 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac26b15-872a-4244-a109-ebe8740b85f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:06.668629 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac26b15-872a-4244-a109-ebe8740b85f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:06.668792 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c2d5d0-5fe0-4f2e-9518-3138a659b16c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:06.668895 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac26b15-872a-4244-a109-ebe8740b85f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:06.669240 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac26b15-872a-4244-a109-ebe8740b85f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:06.670197 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac26b15-872a-4244-a109-ebe8740b85f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:06.671587 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:06.672105 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac26b15-872a-4244-a109-ebe8740b85f8 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:06.675029 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:06.687592 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 816/1597] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:06.688579 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac26b15-872a-4244-a109-ebe8740b85f8 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:06.695583 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 782/1598] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:06.696821 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c2d5d0-5fe0-4f2e-9518-3138a659b16c admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:06.702484 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac26b15-872a-4244-a109-ebe8740b85f8 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:06.707707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6bae1f-858e-46c6-99e9-9de79fbbbc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:06.708530 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6bae1f-858e-46c6-99e9-9de79fbbbc99 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:06.708910 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6bae1f-858e-46c6-99e9-9de79fbbbc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:06.709239 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c2d5d0-5fe0-4f2e-9518-3138a659b16c admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:06.709372 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6bae1f-858e-46c6-99e9-9de79fbbbc99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:06.710099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6bae1f-858e-46c6-99e9-9de79fbbbc99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:06.712040 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6bae1f-858e-46c6-99e9-9de79fbbbc99 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:06.716999 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b6bd8-bc5f-4937-8b0f-7ef5e1ba606b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:06.717866 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b6bd8-bc5f-4937-8b0f-7ef5e1ba606b admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:06.718224 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b6bd8-bc5f-4937-8b0f-7ef5e1ba606b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:06.718620 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b6bd8-bc5f-4937-8b0f-7ef5e1ba606b admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:06.719447 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997b6bd8-bc5f-4937-8b0f-7ef5e1ba606b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:06.721806 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997b6bd8-bc5f-4937-8b0f-7ef5e1ba606b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:06.725252 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 817/1599] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:06.726347 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6bae1f-858e-46c6-99e9-9de79fbbbc99 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:06.739083 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6bae1f-858e-46c6-99e9-9de79fbbbc99 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:06.745192 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b95517f-52af-47a6-89f0-6171447d4af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:06.745997 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b95517f-52af-47a6-89f0-6171447d4af5 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:06.746516 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b95517f-52af-47a6-89f0-6171447d4af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:06.746806 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b95517f-52af-47a6-89f0-6171447d4af5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:06.747559 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b95517f-52af-47a6-89f0-6171447d4af5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:06.749202 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b95517f-52af-47a6-89f0-6171447d4af5 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:06.761006 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 783/1600] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:32:06 2022] POST /identity/v3/users => generated 309 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:06.761006 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b6bd8-bc5f-4937-8b0f-7ef5e1ba606b admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:06.761308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 818/1601] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:06.761308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b95517f-52af-47a6-89f0-6171447d4af5 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:06.771858 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b95517f-52af-47a6-89f0-6171447d4af5 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:06.775913 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b6bd8-bc5f-4937-8b0f-7ef5e1ba606b admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:06.776922 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80b1aee-6b14-4715-b4da-20335cf7f6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:06.777850 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b1aee-6b14-4715-b4da-20335cf7f6a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:06.778209 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b1aee-6b14-4715-b4da-20335cf7f6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:06.778869 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b1aee-6b14-4715-b4da-20335cf7f6a8 admin admin] PATH_INFO: `/v3/projects/8afcda833ca0433e95d6f9de6fc0df21/users/04d032c7e7fb4eaaa9fdf3e92d7e28b8/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:06.783964 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c71e5-aab6-4673-a54d-c8ed77e73a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:06.784917 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c71e5-aab6-4673-a54d-c8ed77e73a34 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:06.785369 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c71e5-aab6-4673-a54d-c8ed77e73a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:06.785870 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c71e5-aab6-4673-a54d-c8ed77e73a34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:06.786901 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52c71e5-aab6-4673-a54d-c8ed77e73a34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:06.789411 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52c71e5-aab6-4673-a54d-c8ed77e73a34 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:06.796785 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80b1aee-6b14-4715-b4da-20335cf7f6a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8afcda833ca0433e95d6f9de6fc0df21, user_id=04d032c7e7fb4eaaa9fdf3e92d7e28b8, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:06.799670 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80b1aee-6b14-4715-b4da-20335cf7f6a8 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:06.804311 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 784/1602] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:06.805217 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c71e5-aab6-4673-a54d-c8ed77e73a34 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:06.822293 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 819/1603] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:06 2022] PUT /identity/v3/projects/8afcda833ca0433e95d6f9de6fc0df21/users/04d032c7e7fb4eaaa9fdf3e92d7e28b8/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:06.822293 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80b1aee-6b14-4715-b4da-20335cf7f6a8 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:06.822691 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c71e5-aab6-4673-a54d-c8ed77e73a34 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:06.897245 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba80bef-2a2e-4100-9d65-3914b073be77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:06.898168 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba80bef-2a2e-4100-9d65-3914b073be77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:06.898645 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba80bef-2a2e-4100-9d65-3914b073be77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:06.898772 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba80bef-2a2e-4100-9d65-3914b073be77 admin admin] PATH_INFO: `/v3/projects/3b4b1280067b4d51ad8de574fdca512b/users/d8fe5b9c070b46f5842be98fe84d6a9d/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:06.917849 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba80bef-2a2e-4100-9d65-3914b073be77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b4b1280067b4d51ad8de574fdca512b, user_id=d8fe5b9c070b46f5842be98fe84d6a9d, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:06.919848 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: WARNING py.warnings [None req-0ba80bef-2a2e-4100-9d65-3914b073be77 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 21:32:06.919848 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: warnings.warn(msg) May 19 21:32:06.919848 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: May 19 21:32:06.923216 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80b1aee-6b14-4715-b4da-20335cf7f6a8 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:06.923679 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba80bef-2a2e-4100-9d65-3914b073be77 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:06.929254 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac049d7-23b3-4acb-a949-f01045cf56b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:06.930597 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac049d7-23b3-4acb-a949-f01045cf56b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:06.931213 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac049d7-23b3-4acb-a949-f01045cf56b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:06.931613 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac049d7-23b3-4acb-a949-f01045cf56b1 admin admin] PATH_INFO: `/v3/projects/985c62c1f3c54f568abdac37360ab457/users/1589968f375946a38e82ec484afc837f/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:06.939393 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 785/1604] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:06 2022] PUT /identity/v3/projects/3b4b1280067b4d51ad8de574fdca512b/users/d8fe5b9c070b46f5842be98fe84d6a9d/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:06.940554 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba80bef-2a2e-4100-9d65-3914b073be77 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:06.957618 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac049d7-23b3-4acb-a949-f01045cf56b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=985c62c1f3c54f568abdac37360ab457, user_id=1589968f375946a38e82ec484afc837f, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:06.961007 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac049d7-23b3-4acb-a949-f01045cf56b1 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:06.982199 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 820/1605] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:06 2022] PUT /identity/v3/projects/985c62c1f3c54f568abdac37360ab457/users/1589968f375946a38e82ec484afc837f/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:06.984029 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac049d7-23b3-4acb-a949-f01045cf56b1 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:07.030896 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba80bef-2a2e-4100-9d65-3914b073be77 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:07.071956 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac049d7-23b3-4acb-a949-f01045cf56b1 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:07.076825 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bbb74c-1444-4548-9643-8436500b1e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:07.077535 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bbb74c-1444-4548-9643-8436500b1e63 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:07.077819 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bbb74c-1444-4548-9643-8436500b1e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:07.078096 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bbb74c-1444-4548-9643-8436500b1e63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:07.079674 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bbb74c-1444-4548-9643-8436500b1e63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:07.080735 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bbb74c-1444-4548-9643-8436500b1e63 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:07.097317 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 821/1606] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:07.098112 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bbb74c-1444-4548-9643-8436500b1e63 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:07.109012 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bbb74c-1444-4548-9643-8436500b1e63 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:07.113639 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514bd247-138c-4160-aac8-a84a104cada4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:07.114616 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514bd247-138c-4160-aac8-a84a104cada4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:07.114994 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514bd247-138c-4160-aac8-a84a104cada4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:07.115362 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514bd247-138c-4160-aac8-a84a104cada4 admin admin] PATH_INFO: `/v3/projects/467d6d8b9b5a443ba9a8f182d3c98a5f/users/79dc156686f948b18944509443c0ab28/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:07.131592 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514bd247-138c-4160-aac8-a84a104cada4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=467d6d8b9b5a443ba9a8f182d3c98a5f, user_id=79dc156686f948b18944509443c0ab28, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:07.134291 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514bd247-138c-4160-aac8-a84a104cada4 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:07.157394 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 822/1607] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:07 2022] PUT /identity/v3/projects/467d6d8b9b5a443ba9a8f182d3c98a5f/users/79dc156686f948b18944509443c0ab28/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:07.157394 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514bd247-138c-4160-aac8-a84a104cada4 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:07.158446 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3abb489-bfc0-446c-af2b-e8c5f28a5bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:07.159935 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3abb489-bfc0-446c-af2b-e8c5f28a5bac admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:07.160406 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3abb489-bfc0-446c-af2b-e8c5f28a5bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:07.160827 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3abb489-bfc0-446c-af2b-e8c5f28a5bac admin admin] PATH_INFO: `/v3/projects/92434c51e27a405aaa89a70a660f6d91/users/f47e7f746a014cb0a35390e7e0ce435d/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:07.171197 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:07.171664 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:07.213079 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3abb489-bfc0-446c-af2b-e8c5f28a5bac admin admin] RBAC: Authorizing `identity:create_grant(project_id=92434c51e27a405aaa89a70a660f6d91, user_id=f47e7f746a014cb0a35390e7e0ce435d, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:07.216076 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3abb489-bfc0-446c-af2b-e8c5f28a5bac admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:07.231609 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 786/1608] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:06 2022] PUT /identity/v3/projects/92434c51e27a405aaa89a70a660f6d91/users/f47e7f746a014cb0a35390e7e0ce435d/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:07.232947 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0303af54-0737-4381-9c15-d6595c65620f None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:07.233233 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0303af54-0737-4381-9c15-d6595c65620f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:07.233508 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0303af54-0737-4381-9c15-d6595c65620f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:07.250971 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514bd247-138c-4160-aac8-a84a104cada4 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:07.264092 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-0303af54-0737-4381-9c15-d6595c65620f None None] MFA Rules not processed for user `04d032c7e7fb4eaaa9fdf3e92d7e28b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:07.265354 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0303af54-0737-4381-9c15-d6595c65620f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:07.324312 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980aa553-cad7-4088-b12b-10b3abcc9c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:07.326035 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980aa553-cad7-4088-b12b-10b3abcc9c9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:07.326443 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980aa553-cad7-4088-b12b-10b3abcc9c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:07.326953 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980aa553-cad7-4088-b12b-10b3abcc9c9b admin admin] PATH_INFO: `/v3/projects/4f5916770e9b443489fe7aa76d663fe1/users/6d16feb169ac46f7983a6b8045de77c9/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:07.341565 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 787/1609] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:07.342897 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d7c33-a4e4-4a42-9097-aaf0f37cbc73 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:07.343185 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d7c33-a4e4-4a42-9097-aaf0f37cbc73 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:07.343464 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d7c33-a4e4-4a42-9097-aaf0f37cbc73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:07.354725 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980aa553-cad7-4088-b12b-10b3abcc9c9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f5916770e9b443489fe7aa76d663fe1, user_id=6d16feb169ac46f7983a6b8045de77c9, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:07.359246 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980aa553-cad7-4088-b12b-10b3abcc9c9b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:07.380187 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-717d7c33-a4e4-4a42-9097-aaf0f37cbc73 None None] MFA Rules not processed for user `d8fe5b9c070b46f5842be98fe84d6a9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:07.381926 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-717d7c33-a4e4-4a42-9097-aaf0f37cbc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:07.382044 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 823/1610] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:07 2022] PUT /identity/v3/projects/4f5916770e9b443489fe7aa76d663fe1/users/6d16feb169ac46f7983a6b8045de77c9/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:07.383561 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1a77b-df38-4a32-b79e-8a3fd0eeae6d None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:07.383919 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1a77b-df38-4a32-b79e-8a3fd0eeae6d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:07.384367 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1a77b-df38-4a32-b79e-8a3fd0eeae6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:07.405352 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-cef1a77b-df38-4a32-b79e-8a3fd0eeae6d None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:07.406653 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cef1a77b-df38-4a32-b79e-8a3fd0eeae6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:07.460916 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 788/1611] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:07 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:07.461938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d7c33-a4e4-4a42-9097-aaf0f37cbc73 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:07.476189 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 824/1612] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:07.479573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cfebce-42f1-4d51-87d4-fb88833d5528 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:07.479573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cfebce-42f1-4d51-87d4-fb88833d5528 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:07.479573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cfebce-42f1-4d51-87d4-fb88833d5528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:07.507378 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b5cfebce-42f1-4d51-87d4-fb88833d5528 None None] MFA Rules not processed for user `79dc156686f948b18944509443c0ab28`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:07.508703 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b5cfebce-42f1-4d51-87d4-fb88833d5528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:07.525830 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d7c33-a4e4-4a42-9097-aaf0f37cbc73 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:07.530485 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39950513-aa3d-4b29-b426-f6de24f47fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:07.531388 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39950513-aa3d-4b29-b426-f6de24f47fc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:07.531683 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39950513-aa3d-4b29-b426-f6de24f47fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:07.531947 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39950513-aa3d-4b29-b426-f6de24f47fc7 admin admin] PATH_INFO: `/v3/projects/0e36f856df8c4a4a9c584370e5c0ad6b/users/e81522f8e00e49b5b12452edf814a999/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:07.535133 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:07.535536 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:07.549705 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39950513-aa3d-4b29-b426-f6de24f47fc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e36f856df8c4a4a9c584370e5c0ad6b, user_id=e81522f8e00e49b5b12452edf814a999, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:07.553105 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39950513-aa3d-4b29-b426-f6de24f47fc7 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:07.567295 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 789/1613] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:07 2022] PUT /identity/v3/projects/0e36f856df8c4a4a9c584370e5c0ad6b/users/e81522f8e00e49b5b12452edf814a999/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:07.568622 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d1aff-fb00-46fd-a997-afbd485cfa1a None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:07.568905 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d1aff-fb00-46fd-a997-afbd485cfa1a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:07.569181 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d1aff-fb00-46fd-a997-afbd485cfa1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:07.578723 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 825/1614] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:07.580066 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f4293-7060-4f09-8e9f-c9c957940652 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:07.580481 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f4293-7060-4f09-8e9f-c9c957940652 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:07.580857 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f4293-7060-4f09-8e9f-c9c957940652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:07.598138 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-6e1d1aff-fb00-46fd-a997-afbd485cfa1a None None] MFA Rules not processed for user `bc570204a2bb4889889a67850dd1d9fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:07.598766 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-5d5f4293-7060-4f09-8e9f-c9c957940652 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:07.599759 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6e1d1aff-fb00-46fd-a997-afbd485cfa1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:07.599868 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5d5f4293-7060-4f09-8e9f-c9c957940652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:07.662608 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 826/1615] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:07.663737 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ccf395-38e5-4a9d-b22c-8c91bfe5a0ef None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:07.664022 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ccf395-38e5-4a9d-b22c-8c91bfe5a0ef None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:07.664316 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ccf395-38e5-4a9d-b22c-8c91bfe5a0ef None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:07.665581 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 827/1616] 10.222.0.65 () {54 vars in 961 bytes} [Thu May 19 21:32:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:32:07.669220 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 790/1617] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:07.670050 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d1aff-fb00-46fd-a997-afbd485cfa1a None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:07.671411 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca841d3d-b372-453c-8787-607c7439f2b5 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:07.671675 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca841d3d-b372-453c-8787-607c7439f2b5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:07.671974 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca841d3d-b372-453c-8787-607c7439f2b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:07.689775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d1aff-fb00-46fd-a997-afbd485cfa1a None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:07.694194 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-ca841d3d-b372-453c-8787-607c7439f2b5 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:07.695414 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ca841d3d-b372-453c-8787-607c7439f2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:07.695531 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4830b2-1054-4a14-b219-fd95afea1f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:07.696212 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4830b2-1054-4a14-b219-fd95afea1f9b admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:07.696496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4830b2-1054-4a14-b219-fd95afea1f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:07.696843 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4830b2-1054-4a14-b219-fd95afea1f9b admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:07.697469 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4830b2-1054-4a14-b219-fd95afea1f9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:07.699473 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4830b2-1054-4a14-b219-fd95afea1f9b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:07.703141 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 828/1618] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:07.703915 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca841d3d-b372-453c-8787-607c7439f2b5 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:07.733497 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 791/1619] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:32:07 2022] POST /identity/v3/projects => generated 367 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:07.734610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4830b2-1054-4a14-b219-fd95afea1f9b admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:07.762600 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca841d3d-b372-453c-8787-607c7439f2b5 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:07.801250 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4830b2-1054-4a14-b219-fd95afea1f9b admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:07.806320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77fd912-8bbd-4c69-9f35-975dc1d81977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:07.807000 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77fd912-8bbd-4c69-9f35-975dc1d81977 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:07.807275 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77fd912-8bbd-4c69-9f35-975dc1d81977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:07.807565 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77fd912-8bbd-4c69-9f35-975dc1d81977 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:07.808303 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77fd912-8bbd-4c69-9f35-975dc1d81977 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:07.809918 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77fd912-8bbd-4c69-9f35-975dc1d81977 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:07.812040 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96505d7-964c-4d50-9d18-a0f18d23d835 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:07.812703 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96505d7-964c-4d50-9d18-a0f18d23d835 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:07.812990 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96505d7-964c-4d50-9d18-a0f18d23d835 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:07.813272 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96505d7-964c-4d50-9d18-a0f18d23d835 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:07.823084 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96505d7-964c-4d50-9d18-a0f18d23d835 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:07.824049 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 792/1620] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:07.824559 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96505d7-964c-4d50-9d18-a0f18d23d835 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:07.824986 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77fd912-8bbd-4c69-9f35-975dc1d81977 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:07.834776 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77fd912-8bbd-4c69-9f35-975dc1d81977 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:07.835427 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 829/1621] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:07.836655 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b1b13-4bc1-4110-9933-621dfea5c082 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:07.836935 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b1b13-4bc1-4110-9933-621dfea5c082 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:07.837213 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b1b13-4bc1-4110-9933-621dfea5c082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:07.839609 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b9315-feae-4bb5-93f7-e6ccb082409c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:07.840268 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b9315-feae-4bb5-93f7-e6ccb082409c admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:07.840555 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b9315-feae-4bb5-93f7-e6ccb082409c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:07.840835 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b9315-feae-4bb5-93f7-e6ccb082409c admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:07.841778 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7b9315-feae-4bb5-93f7-e6ccb082409c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:07.843611 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7b9315-feae-4bb5-93f7-e6ccb082409c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:07.864055 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-174b1b13-4bc1-4110-9933-621dfea5c082 None None] MFA Rules not processed for user `e254671b21fe4661a381b9521810cf80`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:07.866182 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-174b1b13-4bc1-4110-9933-621dfea5c082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:07.877140 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 793/1622] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:32:07 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:07.877140 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b9315-feae-4bb5-93f7-e6ccb082409c admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:07.888084 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:07.918294 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:07.952555 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b9315-feae-4bb5-93f7-e6ccb082409c admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:07.954871 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 830/1623] 10.222.0.65 () {58 vars in 1058 bytes} [Thu May 19 21:32:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:07.956263 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a6d01-8f22-4427-b604-190e1eb75f73 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:07.956734 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a6d01-8f22-4427-b604-190e1eb75f73 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:07.956956 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a6d01-8f22-4427-b604-190e1eb75f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:07.957631 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e4f2a-8c95-49a7-90da-3497bff13076 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:07.958973 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e4f2a-8c95-49a7-90da-3497bff13076 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:07.959371 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e4f2a-8c95-49a7-90da-3497bff13076 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:07.959747 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e4f2a-8c95-49a7-90da-3497bff13076 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:07.978401 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-265a6d01-8f22-4427-b604-190e1eb75f73 None None] MFA Rules not processed for user `bc570204a2bb4889889a67850dd1d9fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:07.979651 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-265a6d01-8f22-4427-b604-190e1eb75f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:08.036205 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369e4f2a-8c95-49a7-90da-3497bff13076 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:08.038067 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369e4f2a-8c95-49a7-90da-3497bff13076 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:08.052057 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 831/1624] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:08.052057 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a6d01-8f22-4427-b604-190e1eb75f73 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:08.054222 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 794/1625] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:08.055455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e4f2a-8c95-49a7-90da-3497bff13076 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:08.061574 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a6d01-8f22-4427-b604-190e1eb75f73 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:08.066959 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b897d1-8f55-4347-9f63-039041ed0381 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:08.067190 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e4f2a-8c95-49a7-90da-3497bff13076 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:08.068188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b897d1-8f55-4347-9f63-039041ed0381 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:08.068546 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b897d1-8f55-4347-9f63-039041ed0381 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:08.068944 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b897d1-8f55-4347-9f63-039041ed0381 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:08.073899 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946cc99-c9d5-4626-b0e1-a5c492b7896e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:08.074738 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946cc99-c9d5-4626-b0e1-a5c492b7896e admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:08.075286 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946cc99-c9d5-4626-b0e1-a5c492b7896e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:08.075988 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946cc99-c9d5-4626-b0e1-a5c492b7896e admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:08.076434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5946cc99-c9d5-4626-b0e1-a5c492b7896e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:08.079069 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5946cc99-c9d5-4626-b0e1-a5c492b7896e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:08.081454 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b897d1-8f55-4347-9f63-039041ed0381 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:08.083523 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b897d1-8f55-4347-9f63-039041ed0381 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:08.101574 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 832/1626] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:08.102632 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b897d1-8f55-4347-9f63-039041ed0381 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:08.112296 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 795/1627] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:32:08 2022] POST /identity/v3/users => generated 290 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:08.113475 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946cc99-c9d5-4626-b0e1-a5c492b7896e admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:08.120217 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b897d1-8f55-4347-9f63-039041ed0381 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:08.125272 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97feae01-a927-49c3-9f03-22ca251ec542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:08.125848 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946cc99-c9d5-4626-b0e1-a5c492b7896e admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:08.127022 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97feae01-a927-49c3-9f03-22ca251ec542 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:08.127484 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97feae01-a927-49c3-9f03-22ca251ec542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:08.127654 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97feae01-a927-49c3-9f03-22ca251ec542 admin admin] PATH_INFO: `/v3/projects/0e36f856df8c4a4a9c584370e5c0ad6b/users/e81522f8e00e49b5b12452edf814a999/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:08.130362 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97feae01-a927-49c3-9f03-22ca251ec542 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e36f856df8c4a4a9c584370e5c0ad6b, user_id=e81522f8e00e49b5b12452edf814a999, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:08.132449 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25591fff-7922-4c49-9feb-a14d43d17129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:08.133256 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25591fff-7922-4c49-9feb-a14d43d17129 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:08.133577 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25591fff-7922-4c49-9feb-a14d43d17129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:08.133688 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97feae01-a927-49c3-9f03-22ca251ec542 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:08.133854 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25591fff-7922-4c49-9feb-a14d43d17129 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:08.134682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25591fff-7922-4c49-9feb-a14d43d17129 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:08.136734 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25591fff-7922-4c49-9feb-a14d43d17129 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:08.150384 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 833/1628] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:08 2022] PUT /identity/v3/projects/0e36f856df8c4a4a9c584370e5c0ad6b/users/e81522f8e00e49b5b12452edf814a999/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:08.150384 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97feae01-a927-49c3-9f03-22ca251ec542 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:08.163021 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 796/1629] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:32:08 2022] POST /identity/v3/users => generated 292 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:08.164858 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9a148-8f78-412d-b664-a0a879bb7b50 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:08.165148 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9a148-8f78-412d-b664-a0a879bb7b50 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:08.165422 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9a148-8f78-412d-b664-a0a879bb7b50 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:08.166816 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 797/1630] 10.222.0.65 () {54 vars in 961 bytes} [Thu May 19 21:32:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:32:08.167504 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9a148-8f78-412d-b664-a0a879bb7b50 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:08.253364 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97feae01-a927-49c3-9f03-22ca251ec542 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:08.260522 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f82e7-f190-4c08-b28b-8daca9d8f318 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:08.261458 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f82e7-f190-4c08-b28b-8daca9d8f318 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:08.261733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f82e7-f190-4c08-b28b-8daca9d8f318 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:08.262020 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f82e7-f190-4c08-b28b-8daca9d8f318 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:08.272950 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9a148-8f78-412d-b664-a0a879bb7b50 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:08.285373 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc9ffe-8c11-4e01-979b-65fc30bf13c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:08.286131 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc9ffe-8c11-4e01-979b-65fc30bf13c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:08.286431 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc9ffe-8c11-4e01-979b-65fc30bf13c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:08.286709 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc9ffe-8c11-4e01-979b-65fc30bf13c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:08.287504 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc9ffe-8c11-4e01-979b-65fc30bf13c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:08.290486 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc9ffe-8c11-4e01-979b-65fc30bf13c9 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:08.311421 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:08.314430 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 798/1631] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:08.314430 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47df70-00d0-40a6-8bca-57f66099c957 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:08.314899 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47df70-00d0-40a6-8bca-57f66099c957 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:08.315089 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47df70-00d0-40a6-8bca-57f66099c957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:08.380772 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f82e7-f190-4c08-b28b-8daca9d8f318 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:08.382445 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f82e7-f190-4c08-b28b-8daca9d8f318 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:08.386426 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-7d47df70-00d0-40a6-8bca-57f66099c957 None None] MFA Rules not processed for user `e81522f8e00e49b5b12452edf814a999`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:08.389243 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7d47df70-00d0-40a6-8bca-57f66099c957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:08.393539 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 834/1632] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:08.394477 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f82e7-f190-4c08-b28b-8daca9d8f318 service nova] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:08.408355 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f82e7-f190-4c08-b28b-8daca9d8f318 service nova] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:08.414393 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1bb4b-22f5-463d-a12b-b9e5f6607f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:08.415079 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1bb4b-22f5-463d-a12b-b9e5f6607f54 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:08.415366 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1bb4b-22f5-463d-a12b-b9e5f6607f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:08.415642 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1bb4b-22f5-463d-a12b-b9e5f6607f54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:08.416978 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a1bb4b-22f5-463d-a12b-b9e5f6607f54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:08.419146 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a1bb4b-22f5-463d-a12b-b9e5f6607f54 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:08.430167 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:08.436665 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 835/1633] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:08.436665 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e849671-aa47-4403-83d9-289807cb9f3b None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:08.436665 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e849671-aa47-4403-83d9-289807cb9f3b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:08.436665 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e849671-aa47-4403-83d9-289807cb9f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:08.463764 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-3e849671-aa47-4403-83d9-289807cb9f3b None None] MFA Rules not processed for user `e254671b21fe4661a381b9521810cf80`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:08.465537 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3e849671-aa47-4403-83d9-289807cb9f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:08.479342 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 836/1634] 10.222.0.65 () {58 vars in 1058 bytes} [Thu May 19 21:32:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:08.480354 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e849671-aa47-4403-83d9-289807cb9f3b None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:08.494685 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e849671-aa47-4403-83d9-289807cb9f3b None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:08.498131 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 799/1635] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:08 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:08.499099 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d47df70-00d0-40a6-8bca-57f66099c957 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:08.504603 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac953b-0463-4a42-980c-32cc08fc2624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:08.506165 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac953b-0463-4a42-980c-32cc08fc2624 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:08.506874 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac953b-0463-4a42-980c-32cc08fc2624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:08.507371 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac953b-0463-4a42-980c-32cc08fc2624 admin admin] PATH_INFO: `/v3/projects/fc1127658625452993e72e10f80b90a2/users/eb46b220318e4260bb2122cc37fc2410/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:08.511482 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d47df70-00d0-40a6-8bca-57f66099c957 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:08.516650 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6f96d0-9e60-4bab-856a-211a1b31d3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:08.517573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f96d0-9e60-4bab-856a-211a1b31d3e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:08.517859 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f96d0-9e60-4bab-856a-211a1b31d3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:08.518328 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f96d0-9e60-4bab-856a-211a1b31d3e6 admin admin] PATH_INFO: `/v3/projects/efff76d18cd741a59f46194ca09135d0/users/9051ab1c4cde40d2998c28570fc31a86/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:08.532204 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ac953b-0463-4a42-980c-32cc08fc2624 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc1127658625452993e72e10f80b90a2, user_id=eb46b220318e4260bb2122cc37fc2410, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:08.536240 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ac953b-0463-4a42-980c-32cc08fc2624 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:08.536632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6f96d0-9e60-4bab-856a-211a1b31d3e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efff76d18cd741a59f46194ca09135d0, user_id=9051ab1c4cde40d2998c28570fc31a86, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:08.539765 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6f96d0-9e60-4bab-856a-211a1b31d3e6 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:08.560075 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 837/1637] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:08 2022] PUT /identity/v3/projects/fc1127658625452993e72e10f80b90a2/users/eb46b220318e4260bb2122cc37fc2410/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:08.560472 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 800/1637] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:08 2022] PUT /identity/v3/projects/efff76d18cd741a59f46194ca09135d0/users/9051ab1c4cde40d2998c28570fc31a86/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:08.561547 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6f96d0-9e60-4bab-856a-211a1b31d3e6 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:08.588784 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac953b-0463-4a42-980c-32cc08fc2624 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:08.653958 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6f96d0-9e60-4bab-856a-211a1b31d3e6 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:08.754171 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b06b28-ea50-4bc3-b9ce-3db54d2bce94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:08.754738 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b06b28-ea50-4bc3-b9ce-3db54d2bce94 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:08.755025 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b06b28-ea50-4bc3-b9ce-3db54d2bce94 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:08.755397 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b06b28-ea50-4bc3-b9ce-3db54d2bce94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:08.760784 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac953b-0463-4a42-980c-32cc08fc2624 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:08.767850 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dceb31-d489-4c7c-b573-cf9f24a37ed4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:08.768831 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dceb31-d489-4c7c-b573-cf9f24a37ed4 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:08.769116 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dceb31-d489-4c7c-b573-cf9f24a37ed4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:08.769450 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dceb31-d489-4c7c-b573-cf9f24a37ed4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:08.836978 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:08.887985 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b06b28-ea50-4bc3-b9ce-3db54d2bce94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:08.891482 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b06b28-ea50-4bc3-b9ce-3db54d2bce94 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:08.905453 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dceb31-d489-4c7c-b573-cf9f24a37ed4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:08.908423 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dceb31-d489-4c7c-b573-cf9f24a37ed4 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:08.925466 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 801/1638] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:08 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:08.930281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b06b28-ea50-4bc3-b9ce-3db54d2bce94 service nova] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:08.955194 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 838/1639] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:08 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:08.957527 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dceb31-d489-4c7c-b573-cf9f24a37ed4 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:09.073984 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:09.128962 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b06b28-ea50-4bc3-b9ce-3db54d2bce94 service nova] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:09.132197 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dceb31-d489-4c7c-b573-cf9f24a37ed4 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:09.142008 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aef387-e0f9-4b2a-94e3-cdac7d4dd3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:09.143061 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aef387-e0f9-4b2a-94e3-cdac7d4dd3d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:09.143061 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aef387-e0f9-4b2a-94e3-cdac7d4dd3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:09.143268 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aef387-e0f9-4b2a-94e3-cdac7d4dd3d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:09.143678 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c1f06-1a53-444b-b04a-7e0b731ec4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:09.144003 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aef387-e0f9-4b2a-94e3-cdac7d4dd3d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:09.144422 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c1f06-1a53-444b-b04a-7e0b731ec4cd admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:09.145155 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c1f06-1a53-444b-b04a-7e0b731ec4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:09.145283 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c1f06-1a53-444b-b04a-7e0b731ec4cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:09.145756 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aef387-e0f9-4b2a-94e3-cdac7d4dd3d5 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:09.146891 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2c1f06-1a53-444b-b04a-7e0b731ec4cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:09.150667 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2c1f06-1a53-444b-b04a-7e0b731ec4cd admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:09.166914 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 802/1640] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:09.172573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f41254-cd7a-489d-bce2-6ff34d12c108 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:09.173166 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f41254-cd7a-489d-bce2-6ff34d12c108 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:09.173626 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f41254-cd7a-489d-bce2-6ff34d12c108 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:09.181984 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 839/1641] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:09.184094 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c1f06-1a53-444b-b04a-7e0b731ec4cd admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:09.184266 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 803/1642] 10.222.0.65 () {54 vars in 912 bytes} [Thu May 19 21:32:09 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:32:09.185858 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f24106-cf1e-4832-98e1-d6537a3f87ec None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:09.190544 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f24106-cf1e-4832-98e1-d6537a3f87ec None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:09.190998 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f24106-cf1e-4832-98e1-d6537a3f87ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:09.204575 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c1f06-1a53-444b-b04a-7e0b731ec4cd admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:09.227737 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a43183-3743-4843-a379-fe1aa9501f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:09.228675 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a43183-3743-4843-a379-fe1aa9501f33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:09.228960 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a43183-3743-4843-a379-fe1aa9501f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:09.229286 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a43183-3743-4843-a379-fe1aa9501f33 admin admin] PATH_INFO: `/v3/projects/fc1127658625452993e72e10f80b90a2/users/eb46b220318e4260bb2122cc37fc2410/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:09.231934 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a43183-3743-4843-a379-fe1aa9501f33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc1127658625452993e72e10f80b90a2, user_id=eb46b220318e4260bb2122cc37fc2410, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:09.237838 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a43183-3743-4843-a379-fe1aa9501f33 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:09.252860 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-14f24106-cf1e-4832-98e1-d6537a3f87ec None None] MFA Rules not processed for user `e254671b21fe4661a381b9521810cf80`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:09.253802 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-14f24106-cf1e-4832-98e1-d6537a3f87ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:09.267788 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 840/1643] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:09 2022] PUT /identity/v3/projects/fc1127658625452993e72e10f80b90a2/users/eb46b220318e4260bb2122cc37fc2410/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:09.269953 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a43183-3743-4843-a379-fe1aa9501f33 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:09.414125 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 804/1644] 10.222.0.65 () {58 vars in 1009 bytes} [Thu May 19 21:32:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:09.415625 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ce025-d76f-4bc8-98dd-41b6e4b1ef7a None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:09.415917 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ce025-d76f-4bc8-98dd-41b6e4b1ef7a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:09.416188 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ce025-d76f-4bc8-98dd-41b6e4b1ef7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:09.420172 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a43183-3743-4843-a379-fe1aa9501f33 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:09.432980 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96198cbb-4dcd-4840-972f-76ca40654f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:09.433891 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96198cbb-4dcd-4840-972f-76ca40654f9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:09.434270 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96198cbb-4dcd-4840-972f-76ca40654f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:09.434645 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96198cbb-4dcd-4840-972f-76ca40654f9b admin admin] PATH_INFO: `/v3/projects/efff76d18cd741a59f46194ca09135d0/users/9051ab1c4cde40d2998c28570fc31a86/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:09.443286 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96198cbb-4dcd-4840-972f-76ca40654f9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=efff76d18cd741a59f46194ca09135d0, user_id=9051ab1c4cde40d2998c28570fc31a86, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:09.446094 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96198cbb-4dcd-4840-972f-76ca40654f9b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:09.474231 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 841/1645] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:09 2022] PUT /identity/v3/projects/efff76d18cd741a59f46194ca09135d0/users/9051ab1c4cde40d2998c28570fc31a86/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:09.480595 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-5e4ce025-d76f-4bc8-98dd-41b6e4b1ef7a None None] MFA Rules not processed for user `eb46b220318e4260bb2122cc37fc2410`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:09.481804 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5e4ce025-d76f-4bc8-98dd-41b6e4b1ef7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:09.485437 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96198cbb-4dcd-4840-972f-76ca40654f9b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:09.583955 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:09.604431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96198cbb-4dcd-4840-972f-76ca40654f9b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:09.610473 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c1c231-cda5-4d1b-bb2f-337ee93ad54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:09.611788 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1c231-cda5-4d1b-bb2f-337ee93ad54a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:09.612074 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1c231-cda5-4d1b-bb2f-337ee93ad54a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:09.612436 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1c231-cda5-4d1b-bb2f-337ee93ad54a admin admin] PATH_INFO: `/v3/users/aa22d9b1a5734acc889ccda089459177` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:09.613762 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c1c231-cda5-4d1b-bb2f-337ee93ad54a admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa22d9b1a5734acc889ccda089459177)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:09.616018 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c1c231-cda5-4d1b-bb2f-337ee93ad54a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:09.628008 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 805/1646] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:09 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:09.628008 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ce025-d76f-4bc8-98dd-41b6e4b1ef7a None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:09.716298 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-34c1c231-cda5-4d1b-bb2f-337ee93ad54a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa22d9b1a5734acc889ccda089459177'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:09.731560 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:09.746018 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-34c1c231-cda5-4d1b-bb2f-337ee93ad54a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa22d9b1a5734acc889ccda089459177'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:09.748384 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-34c1c231-cda5-4d1b-bb2f-337ee93ad54a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa22d9b1a5734acc889ccda089459177'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:09.750212 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-34c1c231-cda5-4d1b-bb2f-337ee93ad54a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa22d9b1a5734acc889ccda089459177'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:09.755256 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ce025-d76f-4bc8-98dd-41b6e4b1ef7a None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:09.763226 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-34c1c231-cda5-4d1b-bb2f-337ee93ad54a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa22d9b1a5734acc889ccda089459177'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:09.767340 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81389047-d071-4700-8124-82df62fbf0eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:09.768070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81389047-d071-4700-8124-82df62fbf0eb service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:09.768355 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81389047-d071-4700-8124-82df62fbf0eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:09.768627 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81389047-d071-4700-8124-82df62fbf0eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:09.778270 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-81389047-d071-4700-8124-82df62fbf0eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:09.797250 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 842/1647] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:32:09 2022] DELETE /identity/v3/users/aa22d9b1a5734acc889ccda089459177 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:09.803429 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a2f4d-c3b3-464e-8a95-34ec7e57aee7 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:09.803729 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a2f4d-c3b3-464e-8a95-34ec7e57aee7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:09.803997 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a2f4d-c3b3-464e-8a95-34ec7e57aee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:09.869302 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d46a2f4d-c3b3-464e-8a95-34ec7e57aee7 None None] MFA Rules not processed for user `9051ab1c4cde40d2998c28570fc31a86`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:09.870563 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d46a2f4d-c3b3-464e-8a95-34ec7e57aee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:09.891857 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81389047-d071-4700-8124-82df62fbf0eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:09.893951 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81389047-d071-4700-8124-82df62fbf0eb service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:09.910337 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 806/1648] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:09.912817 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef2884-2462-4de2-9b1e-2d7b5aa21852 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:09.913113 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef2884-2462-4de2-9b1e-2d7b5aa21852 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:09.914452 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef2884-2462-4de2-9b1e-2d7b5aa21852 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:09.915946 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 807/1649] 10.222.0.65 () {54 vars in 912 bytes} [Thu May 19 21:32:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:32:09.916732 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef2884-2462-4de2-9b1e-2d7b5aa21852 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:09.918511 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-73ef2884-2462-4de2-9b1e-2d7b5aa21852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:10.061124 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef2884-2462-4de2-9b1e-2d7b5aa21852 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:10.066215 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 843/1650] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:09 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:10.067977 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd19f30-f647-444e-a0c0-e313edc92255 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:10.068273 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd19f30-f647-444e-a0c0-e313edc92255 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:10.068557 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd19f30-f647-444e-a0c0-e313edc92255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:10.073087 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dfae03-b902-49bf-9c8b-61b13ccddb96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:10.073763 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dfae03-b902-49bf-9c8b-61b13ccddb96 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:10.074142 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dfae03-b902-49bf-9c8b-61b13ccddb96 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:10.074381 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dfae03-b902-49bf-9c8b-61b13ccddb96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:10.078374 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a3dfae03-b902-49bf-9c8b-61b13ccddb96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:10.129051 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-8cd19f30-f647-444e-a0c0-e313edc92255 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:10.143169 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:10.154786 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8cd19f30-f647-444e-a0c0-e313edc92255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:10.186774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dfae03-b902-49bf-9c8b-61b13ccddb96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:10.188516 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dfae03-b902-49bf-9c8b-61b13ccddb96 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:10.211101 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 808/1651] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:09 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:10.215123 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ba619-e301-4f68-949c-5318e3a51528 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:10.216837 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ba619-e301-4f68-949c-5318e3a51528 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:10.217510 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ba619-e301-4f68-949c-5318e3a51528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:10.275508 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-ff9ba619-e301-4f68-949c-5318e3a51528 None None] MFA Rules not processed for user `e254671b21fe4661a381b9521810cf80`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:10.281063 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ff9ba619-e301-4f68-949c-5318e3a51528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:10.292921 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 844/1652] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:10.337975 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd19f30-f647-444e-a0c0-e313edc92255 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:10.350338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd19f30-f647-444e-a0c0-e313edc92255 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:10.364246 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6902fcf-cb12-4f77-8ab5-a2a971630f1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:10.365068 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6902fcf-cb12-4f77-8ab5-a2a971630f1e service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:10.365454 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6902fcf-cb12-4f77-8ab5-a2a971630f1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:10.365831 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6902fcf-cb12-4f77-8ab5-a2a971630f1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:10.384683 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6902fcf-cb12-4f77-8ab5-a2a971630f1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:10.386668 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6902fcf-cb12-4f77-8ab5-a2a971630f1e service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:10.412553 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 845/1653] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:10 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:10.414804 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6902fcf-cb12-4f77-8ab5-a2a971630f1e service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:10.416920 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d6902fcf-cb12-4f77-8ab5-a2a971630f1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:10.463298 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 809/1654] 10.222.0.65 () {58 vars in 1009 bytes} [Thu May 19 21:32:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:10.481572 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ba619-e301-4f68-949c-5318e3a51528 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:10.483353 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ff9ba619-e301-4f68-949c-5318e3a51528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:10.595769 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:10.644647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6902fcf-cb12-4f77-8ab5-a2a971630f1e service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:10.659626 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eecb15e-71d2-4f8b-b45c-b7de2b3e1e6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:10.660528 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eecb15e-71d2-4f8b-b45c-b7de2b3e1e6f service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:10.660947 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eecb15e-71d2-4f8b-b45c-b7de2b3e1e6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:10.661327 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eecb15e-71d2-4f8b-b45c-b7de2b3e1e6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:10.679623 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eecb15e-71d2-4f8b-b45c-b7de2b3e1e6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:10.681826 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eecb15e-71d2-4f8b-b45c-b7de2b3e1e6f service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:10.706029 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 846/1655] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:10.713365 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eecb15e-71d2-4f8b-b45c-b7de2b3e1e6f service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:10.734256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eecb15e-71d2-4f8b-b45c-b7de2b3e1e6f service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:10.741448 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd63b5-a912-407e-8467-b0db5b4892ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:10.742232 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd63b5-a912-407e-8467-b0db5b4892ae service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:10.742550 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd63b5-a912-407e-8467-b0db5b4892ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:10.742834 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd63b5-a912-407e-8467-b0db5b4892ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:10.750630 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-76dd63b5-a912-407e-8467-b0db5b4892ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:10.759928 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ba619-e301-4f68-949c-5318e3a51528 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:10.771468 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181b336a-3e2a-4eac-a583-9bcf24a288db service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea874ca4088d4e6c83c9d0497323cf6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:10.772974 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b336a-3e2a-4eac-a583-9bcf24a288db service glance] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:10.772974 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b336a-3e2a-4eac-a583-9bcf24a288db service glance] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:10.773171 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b336a-3e2a-4eac-a583-9bcf24a288db service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:10.812586 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181b336a-3e2a-4eac-a583-9bcf24a288db service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:10.816267 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181b336a-3e2a-4eac-a583-9bcf24a288db service glance] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:10.851660 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 810/1656] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:32:10.942685 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dd63b5-a912-407e-8467-b0db5b4892ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:10.945282 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dd63b5-a912-407e-8467-b0db5b4892ae service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:10.980982 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 847/1657] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:10 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:11.812412 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3582fac7-a3b2-41d2-82f5-98fab2c7e5a1 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:11.813905 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3582fac7-a3b2-41d2-82f5-98fab2c7e5a1 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:11.814809 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3582fac7-a3b2-41d2-82f5-98fab2c7e5a1 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:11.818687 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 811/1658] 10.222.0.65 () {54 vars in 912 bytes} [Thu May 19 21:32:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:32:11.828181 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896d905-d5a8-47a3-bce0-5e5591e35a0e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:11.828977 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896d905-d5a8-47a3-bce0-5e5591e35a0e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:11.829493 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896d905-d5a8-47a3-bce0-5e5591e35a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:11.969056 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c896d905-d5a8-47a3-bce0-5e5591e35a0e None None] MFA Rules not processed for user `2201e66caa4b4a57be09d845d7cb0023`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:11.976312 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c896d905-d5a8-47a3-bce0-5e5591e35a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:11.994977 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 848/1659] 10.222.0.65 () {58 vars in 1009 bytes} [Thu May 19 21:32:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:12.120258 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3582fac7-a3b2-41d2-82f5-98fab2c7e5a1 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:12.138313 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3582fac7-a3b2-41d2-82f5-98fab2c7e5a1 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:12.146302 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54ad86-a0c8-495b-b092-d84b308c1c71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:12.146987 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54ad86-a0c8-495b-b092-d84b308c1c71 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:12.147274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54ad86-a0c8-495b-b092-d84b308c1c71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:12.147551 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54ad86-a0c8-495b-b092-d84b308c1c71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:12.168857 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab54ad86-a0c8-495b-b092-d84b308c1c71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:12.170722 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab54ad86-a0c8-495b-b092-d84b308c1c71 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:12.191032 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c896d905-d5a8-47a3-bce0-5e5591e35a0e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:12.191353 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 812/1660] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:12.195767 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c896d905-d5a8-47a3-bce0-5e5591e35a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:12.215729 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c896d905-d5a8-47a3-bce0-5e5591e35a0e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:12.222218 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0275a2e1-9172-4411-a165-0c50b6665d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:12.223574 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275a2e1-9172-4411-a165-0c50b6665d93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:12.224069 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275a2e1-9172-4411-a165-0c50b6665d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:12.224539 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275a2e1-9172-4411-a165-0c50b6665d93 admin admin] PATH_INFO: `/v3/users/79dc156686f948b18944509443c0ab28` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:12.226069 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0275a2e1-9172-4411-a165-0c50b6665d93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79dc156686f948b18944509443c0ab28)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:12.228327 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0275a2e1-9172-4411-a165-0c50b6665d93 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:12.330739 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0275a2e1-9172-4411-a165-0c50b6665d93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79dc156686f948b18944509443c0ab28'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:12.350023 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0275a2e1-9172-4411-a165-0c50b6665d93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79dc156686f948b18944509443c0ab28'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:12.350900 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0275a2e1-9172-4411-a165-0c50b6665d93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79dc156686f948b18944509443c0ab28'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:12.351610 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0275a2e1-9172-4411-a165-0c50b6665d93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79dc156686f948b18944509443c0ab28'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:12.380114 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0275a2e1-9172-4411-a165-0c50b6665d93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79dc156686f948b18944509443c0ab28'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:12.418946 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 849/1661] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:32:12 2022] DELETE /identity/v3/users/79dc156686f948b18944509443c0ab28 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:12.423450 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c698eaa-b477-483d-9a64-603d4e3dcc67 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:12.424037 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c698eaa-b477-483d-9a64-603d4e3dcc67 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:12.424502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c698eaa-b477-483d-9a64-603d4e3dcc67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:12.460263 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-5c698eaa-b477-483d-9a64-603d4e3dcc67 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:12.461925 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5c698eaa-b477-483d-9a64-603d4e3dcc67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:12.560119 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:12.575696 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 813/1662] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:12.600978 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0275a2e1-9172-4411-a165-0c50b6665d93 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:12.602472 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0275a2e1-9172-4411-a165-0c50b6665d93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:12.666456 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:12.724792 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0275a2e1-9172-4411-a165-0c50b6665d93 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:12.729359 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49460a51-7439-4883-b5bb-6d08c2c7ad14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:12.730214 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49460a51-7439-4883-b5bb-6d08c2c7ad14 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:12.730630 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49460a51-7439-4883-b5bb-6d08c2c7ad14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:12.730975 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49460a51-7439-4883-b5bb-6d08c2c7ad14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:12.743076 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49460a51-7439-4883-b5bb-6d08c2c7ad14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:12.745319 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49460a51-7439-4883-b5bb-6d08c2c7ad14 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:12.758876 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 850/1663] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:12.941953 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c698eaa-b477-483d-9a64-603d4e3dcc67 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:12.943534 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5c698eaa-b477-483d-9a64-603d4e3dcc67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:12.981313 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c698eaa-b477-483d-9a64-603d4e3dcc67 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:13.003366 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1078c12a-9184-4cd8-9e59-b9a68b26e542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:13.009654 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078c12a-9184-4cd8-9e59-b9a68b26e542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:13.010281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078c12a-9184-4cd8-9e59-b9a68b26e542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:13.010678 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078c12a-9184-4cd8-9e59-b9a68b26e542 admin admin] PATH_INFO: `/v3/projects/c6babff9bc984c8f9ad6ffa24ea4b507` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:13.020596 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1078c12a-9184-4cd8-9e59-b9a68b26e542 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6babff9bc984c8f9ad6ffa24ea4b507)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:13.031425 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1078c12a-9184-4cd8-9e59-b9a68b26e542 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:13.214996 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1078c12a-9184-4cd8-9e59-b9a68b26e542 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6babff9bc984c8f9ad6ffa24ea4b507', 'request_id': 'req-1078c12a-9184-4cd8-9e59-b9a68b26e542'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:13.255682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1078c12a-9184-4cd8-9e59-b9a68b26e542 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6babff9bc984c8f9ad6ffa24ea4b507', 'request_id': 'req-1078c12a-9184-4cd8-9e59-b9a68b26e542'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:13.289520 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1078c12a-9184-4cd8-9e59-b9a68b26e542 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6babff9bc984c8f9ad6ffa24ea4b507', 'request_id': 'req-1078c12a-9184-4cd8-9e59-b9a68b26e542'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:13.324700 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1078c12a-9184-4cd8-9e59-b9a68b26e542 admin admin] The token cache is being invalidate because project c6babff9bc984c8f9ad6ffa24ea4b507 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:32:13.324700 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1078c12a-9184-4cd8-9e59-b9a68b26e542 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:13.327647 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1078c12a-9184-4cd8-9e59-b9a68b26e542 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:13.332492 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 814/1664] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:32:12 2022] DELETE /identity/v3/projects/c6babff9bc984c8f9ad6ffa24ea4b507 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:13.336634 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49460a51-7439-4883-b5bb-6d08c2c7ad14 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:13.352744 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49460a51-7439-4883-b5bb-6d08c2c7ad14 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:13.516248 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9f6c2-5ef2-40d0-befa-014be02b7495 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:13.516912 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9f6c2-5ef2-40d0-befa-014be02b7495 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:13.517298 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9f6c2-5ef2-40d0-befa-014be02b7495 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:13.524928 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 815/1665] 10.222.0.65 () {54 vars in 960 bytes} [Thu May 19 21:32:13 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:32:13.530639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07cc68-8dc7-4dfb-bd2a-3bf111db5a06 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:13.531194 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07cc68-8dc7-4dfb-bd2a-3bf111db5a06 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:13.531710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07cc68-8dc7-4dfb-bd2a-3bf111db5a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:13.570567 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-8e07cc68-8dc7-4dfb-bd2a-3bf111db5a06 None None] MFA Rules not processed for user `f6143bddc05a46ea972709576c7ad95f`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:13.572101 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8e07cc68-8dc7-4dfb-bd2a-3bf111db5a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:13.652323 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444fdf8-19a0-4cc3-906d-7b46c7f9d0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:13.653228 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444fdf8-19a0-4cc3-906d-7b46c7f9d0c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:13.653510 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444fdf8-19a0-4cc3-906d-7b46c7f9d0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:13.653787 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444fdf8-19a0-4cc3-906d-7b46c7f9d0c0 admin admin] PATH_INFO: `/v3/users/e81522f8e00e49b5b12452edf814a999` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:13.657862 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444fdf8-19a0-4cc3-906d-7b46c7f9d0c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e81522f8e00e49b5b12452edf814a999)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:13.669213 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444fdf8-19a0-4cc3-906d-7b46c7f9d0c0 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:13.765642 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 816/1666] 10.222.0.65 () {58 vars in 1057 bytes} [Thu May 19 21:32:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:32:13.783022 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e07cc68-8dc7-4dfb-bd2a-3bf111db5a06 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:13.791082 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6444fdf8-19a0-4cc3-906d-7b46c7f9d0c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e81522f8e00e49b5b12452edf814a999'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:13.808582 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e07cc68-8dc7-4dfb-bd2a-3bf111db5a06 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:13.824278 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6444fdf8-19a0-4cc3-906d-7b46c7f9d0c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e81522f8e00e49b5b12452edf814a999'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:13.825292 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e327b391-a7ee-40c9-8d98-c4f22be952f9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6143bddc05a46ea972709576c7ad95f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:13.826022 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327b391-a7ee-40c9-8d98-c4f22be952f9 service placement] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:13.826738 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6444fdf8-19a0-4cc3-906d-7b46c7f9d0c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e81522f8e00e49b5b12452edf814a999'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:13.827126 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327b391-a7ee-40c9-8d98-c4f22be952f9 service placement] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:13.827414 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327b391-a7ee-40c9-8d98-c4f22be952f9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:13.828314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6444fdf8-19a0-4cc3-906d-7b46c7f9d0c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e81522f8e00e49b5b12452edf814a999'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:13.831905 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e327b391-a7ee-40c9-8d98-c4f22be952f9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:13.853724 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6444fdf8-19a0-4cc3-906d-7b46c7f9d0c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e81522f8e00e49b5b12452edf814a999'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:13.861749 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e327b391-a7ee-40c9-8d98-c4f22be952f9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:13.863610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e327b391-a7ee-40c9-8d98-c4f22be952f9 service placement] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:13.876437 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 851/1667] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:32:13 2022] DELETE /identity/v3/users/e81522f8e00e49b5b12452edf814a999 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:13.891462 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 817/1668] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:14.914762 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444fdf8-19a0-4cc3-906d-7b46c7f9d0c0 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:14.926774 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6444fdf8-19a0-4cc3-906d-7b46c7f9d0c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:15.159153 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444fdf8-19a0-4cc3-906d-7b46c7f9d0c0 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:15.169506 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe03fd-dc7c-4673-88f3-d54be4969996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:15.173210 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe03fd-dc7c-4673-88f3-d54be4969996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:15.174659 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe03fd-dc7c-4673-88f3-d54be4969996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:15.174968 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe03fd-dc7c-4673-88f3-d54be4969996 admin admin] PATH_INFO: `/v3/projects/467d6d8b9b5a443ba9a8f182d3c98a5f` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:15.177363 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fe03fd-dc7c-4673-88f3-d54be4969996 admin admin] RBAC: Authorizing `identity:delete_project(project_id=467d6d8b9b5a443ba9a8f182d3c98a5f)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:15.182003 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: WARNING py.warnings [None req-74fe03fd-dc7c-4673-88f3-d54be4969996 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 21:32:15.182003 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: warnings.warn(msg) May 19 21:32:15.182003 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: May 19 21:32:15.187075 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fe03fd-dc7c-4673-88f3-d54be4969996 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:15.191468 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:15.330205 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-74fe03fd-dc7c-4673-88f3-d54be4969996 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '467d6d8b9b5a443ba9a8f182d3c98a5f', 'request_id': 'req-74fe03fd-dc7c-4673-88f3-d54be4969996'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:15.386841 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-74fe03fd-dc7c-4673-88f3-d54be4969996 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '467d6d8b9b5a443ba9a8f182d3c98a5f', 'request_id': 'req-74fe03fd-dc7c-4673-88f3-d54be4969996'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:15.416396 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-74fe03fd-dc7c-4673-88f3-d54be4969996 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '467d6d8b9b5a443ba9a8f182d3c98a5f', 'request_id': 'req-74fe03fd-dc7c-4673-88f3-d54be4969996'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:15.457599 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-74fe03fd-dc7c-4673-88f3-d54be4969996 admin admin] The token cache is being invalidate because project 467d6d8b9b5a443ba9a8f182d3c98a5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:32:15.458166 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-74fe03fd-dc7c-4673-88f3-d54be4969996 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:15.460876 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-74fe03fd-dc7c-4673-88f3-d54be4969996 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:15.473076 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 852/1669] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:32:14 2022] DELETE /identity/v3/projects/467d6d8b9b5a443ba9a8f182d3c98a5f => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:15.474544 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e327b391-a7ee-40c9-8d98-c4f22be952f9 service placement] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:15.511873 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e327b391-a7ee-40c9-8d98-c4f22be952f9 service placement] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:15.713208 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07e43a-caea-440c-941a-bb9d0af763fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:15.714140 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07e43a-caea-440c-941a-bb9d0af763fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:15.714451 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07e43a-caea-440c-941a-bb9d0af763fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:15.714874 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07e43a-caea-440c-941a-bb9d0af763fd admin admin] PATH_INFO: `/v3/users/9051ab1c4cde40d2998c28570fc31a86` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:15.716352 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e07e43a-caea-440c-941a-bb9d0af763fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=9051ab1c4cde40d2998c28570fc31a86)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:15.717891 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: WARNING py.warnings [None req-9e07e43a-caea-440c-941a-bb9d0af763fd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 19 21:32:15.717891 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: warnings.warn(msg) May 19 21:32:15.717891 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: May 19 21:32:15.719870 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e07e43a-caea-440c-941a-bb9d0af763fd admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:15.748863 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:15.915234 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9e07e43a-caea-440c-941a-bb9d0af763fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9051ab1c4cde40d2998c28570fc31a86'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:15.916248 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9e07e43a-caea-440c-941a-bb9d0af763fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9051ab1c4cde40d2998c28570fc31a86'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:15.973989 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9e07e43a-caea-440c-941a-bb9d0af763fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9051ab1c4cde40d2998c28570fc31a86'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:16.003638 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9e07e43a-caea-440c-941a-bb9d0af763fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9051ab1c4cde40d2998c28570fc31a86'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:16.019752 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9e07e43a-caea-440c-941a-bb9d0af763fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9051ab1c4cde40d2998c28570fc31a86'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:16.036088 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 818/1670] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:32:15 2022] DELETE /identity/v3/users/9051ab1c4cde40d2998c28570fc31a86 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:16.288527 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe03fd-dc7c-4673-88f3-d54be4969996 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:16.291793 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-74fe03fd-dc7c-4673-88f3-d54be4969996 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:16.335524 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe03fd-dc7c-4673-88f3-d54be4969996 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:16.495785 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e169e-357c-4325-b593-c42bd00a8a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:16.497246 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e169e-357c-4325-b593-c42bd00a8a6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:16.497593 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e169e-357c-4325-b593-c42bd00a8a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:16.497993 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e169e-357c-4325-b593-c42bd00a8a6a admin admin] PATH_INFO: `/v3/projects/0e36f856df8c4a4a9c584370e5c0ad6b` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:16.500223 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e169e-357c-4325-b593-c42bd00a8a6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e36f856df8c4a4a9c584370e5c0ad6b)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:16.503780 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e169e-357c-4325-b593-c42bd00a8a6a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:16.646614 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7b1e169e-357c-4325-b593-c42bd00a8a6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e36f856df8c4a4a9c584370e5c0ad6b', 'request_id': 'req-7b1e169e-357c-4325-b593-c42bd00a8a6a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:16.667101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7b1e169e-357c-4325-b593-c42bd00a8a6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e36f856df8c4a4a9c584370e5c0ad6b', 'request_id': 'req-7b1e169e-357c-4325-b593-c42bd00a8a6a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:16.688613 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7b1e169e-357c-4325-b593-c42bd00a8a6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e36f856df8c4a4a9c584370e5c0ad6b', 'request_id': 'req-7b1e169e-357c-4325-b593-c42bd00a8a6a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:16.727893 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07e43a-caea-440c-941a-bb9d0af763fd admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:16.729485 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9e07e43a-caea-440c-941a-bb9d0af763fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:16.745084 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7b1e169e-357c-4325-b593-c42bd00a8a6a admin admin] The token cache is being invalidate because project 0e36f856df8c4a4a9c584370e5c0ad6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:32:16.746793 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7b1e169e-357c-4325-b593-c42bd00a8a6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:16.749776 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7b1e169e-357c-4325-b593-c42bd00a8a6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:16.759823 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07e43a-caea-440c-941a-bb9d0af763fd admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:16.762913 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 853/1671] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:32:16 2022] DELETE /identity/v3/projects/0e36f856df8c4a4a9c584370e5c0ad6b => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:16.769311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22630707-e900-4944-9f6b-04ba7b80e0f1 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:16.769939 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22630707-e900-4944-9f6b-04ba7b80e0f1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:16.770413 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22630707-e900-4944-9f6b-04ba7b80e0f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:16.816038 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-22630707-e900-4944-9f6b-04ba7b80e0f1 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:16.820207 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-22630707-e900-4944-9f6b-04ba7b80e0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:16.899362 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:16.985317 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf67b1a1-2cf7-4310-9cc2-c5fee6c67f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:16.986808 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67b1a1-2cf7-4310-9cc2-c5fee6c67f9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:16.987205 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67b1a1-2cf7-4310-9cc2-c5fee6c67f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:16.993691 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67b1a1-2cf7-4310-9cc2-c5fee6c67f9f admin admin] PATH_INFO: `/v3/users/d8fe5b9c070b46f5842be98fe84d6a9d` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:16.997276 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf67b1a1-2cf7-4310-9cc2-c5fee6c67f9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8fe5b9c070b46f5842be98fe84d6a9d)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:16.999937 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf67b1a1-2cf7-4310-9cc2-c5fee6c67f9f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:17.029158 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 854/1672] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:17.037295 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303daf7f-1bae-4841-87d1-4f6ccfb94b34 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:17.037940 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303daf7f-1bae-4841-87d1-4f6ccfb94b34 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:17.038381 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303daf7f-1bae-4841-87d1-4f6ccfb94b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:17.118996 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-303daf7f-1bae-4841-87d1-4f6ccfb94b34 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:17.120468 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-303daf7f-1bae-4841-87d1-4f6ccfb94b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:17.131377 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bf67b1a1-2cf7-4310-9cc2-c5fee6c67f9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8fe5b9c070b46f5842be98fe84d6a9d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:17.132592 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bf67b1a1-2cf7-4310-9cc2-c5fee6c67f9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8fe5b9c070b46f5842be98fe84d6a9d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:17.166017 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 855/1673] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:32:17.166968 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bf67b1a1-2cf7-4310-9cc2-c5fee6c67f9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8fe5b9c070b46f5842be98fe84d6a9d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:17.171617 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303daf7f-1bae-4841-87d1-4f6ccfb94b34 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:17.187384 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bf67b1a1-2cf7-4310-9cc2-c5fee6c67f9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8fe5b9c070b46f5842be98fe84d6a9d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:17.201191 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303daf7f-1bae-4841-87d1-4f6ccfb94b34 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:17.204309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bf67b1a1-2cf7-4310-9cc2-c5fee6c67f9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8fe5b9c070b46f5842be98fe84d6a9d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:17.217844 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 819/1674] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:32:16 2022] DELETE /identity/v3/users/d8fe5b9c070b46f5842be98fe84d6a9d => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:17.225258 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf71ec4-b517-4a3a-8097-502229440eb7 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:17.225991 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf71ec4-b517-4a3a-8097-502229440eb7 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:17.226988 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf71ec4-b517-4a3a-8097-502229440eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:17.238029 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c12f1a-237b-414d-b31e-5db2d8a4273a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:17.239034 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c12f1a-237b-414d-b31e-5db2d8a4273a admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:17.239439 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c12f1a-237b-414d-b31e-5db2d8a4273a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:17.239838 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c12f1a-237b-414d-b31e-5db2d8a4273a admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:17.240730 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c12f1a-237b-414d-b31e-5db2d8a4273a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:32:17.241045 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c12f1a-237b-414d-b31e-5db2d8a4273a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:17.242721 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c12f1a-237b-414d-b31e-5db2d8a4273a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:17.262597 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:17.276724 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 856/1675] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:32:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:32:17.281466 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c12f1a-237b-414d-b31e-5db2d8a4273a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:17.283589 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-69c12f1a-237b-414d-b31e-5db2d8a4273a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:17.304547 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-2cf71ec4-b517-4a3a-8097-502229440eb7 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:17.305844 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-2cf71ec4-b517-4a3a-8097-502229440eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:17.316049 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c12f1a-237b-414d-b31e-5db2d8a4273a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:17.461692 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b2442e-b966-4157-849e-3c1b4b7dfbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:17.462590 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2442e-b966-4157-849e-3c1b4b7dfbec admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:17.462795 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2442e-b966-4157-849e-3c1b4b7dfbec admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:17.463074 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2442e-b966-4157-849e-3c1b4b7dfbec admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:17.463790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b2442e-b966-4157-849e-3c1b4b7dfbec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:17.465768 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b2442e-b966-4157-849e-3c1b4b7dfbec admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:17.474306 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 820/1676] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:17.540773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf71ec4-b517-4a3a-8097-502229440eb7 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:17.542138 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 857/1677] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:32:17 2022] POST /identity/v3/projects => generated 367 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:17.542953 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-2cf71ec4-b517-4a3a-8097-502229440eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:17.547589 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b2442e-b966-4157-849e-3c1b4b7dfbec admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:17.709538 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf71ec4-b517-4a3a-8097-502229440eb7 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:17.725142 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b2442e-b966-4157-849e-3c1b4b7dfbec admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:17.743411 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762be69-13bd-4437-b6b5-0717dbf3b647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:17.749252 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762be69-13bd-4437-b6b5-0717dbf3b647 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:17.750061 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762be69-13bd-4437-b6b5-0717dbf3b647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:17.750061 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762be69-13bd-4437-b6b5-0717dbf3b647 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:17.750571 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762be69-13bd-4437-b6b5-0717dbf3b647 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:17.758214 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762be69-13bd-4437-b6b5-0717dbf3b647 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:17.810824 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 858/1678] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:32:17 2022] POST /identity/v3/users => generated 284 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:17.815124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762be69-13bd-4437-b6b5-0717dbf3b647 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:17.828218 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18f0b8-e663-42d3-b5b8-8d5d5b621bf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:17.829745 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18f0b8-e663-42d3-b5b8-8d5d5b621bf2 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:17.831542 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18f0b8-e663-42d3-b5b8-8d5d5b621bf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:17.832282 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18f0b8-e663-42d3-b5b8-8d5d5b621bf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:17.843235 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762be69-13bd-4437-b6b5-0717dbf3b647 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:17.850467 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac18f0b8-e663-42d3-b5b8-8d5d5b621bf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:17.851647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5251b55-7798-4583-9509-472d19f0fb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:17.852333 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5251b55-7798-4583-9509-472d19f0fb1a admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:17.856556 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5251b55-7798-4583-9509-472d19f0fb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:17.856556 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5251b55-7798-4583-9509-472d19f0fb1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:17.856556 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5251b55-7798-4583-9509-472d19f0fb1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:17.856556 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5251b55-7798-4583-9509-472d19f0fb1a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:17.856836 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac18f0b8-e663-42d3-b5b8-8d5d5b621bf2 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:17.880388 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 859/1679] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 21:32:17.883493 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5251b55-7798-4583-9509-472d19f0fb1a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:17.885794 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 821/1680] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:17.895346 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:17.904894 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5251b55-7798-4583-9509-472d19f0fb1a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:17.914931 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e9956b-d52b-4e62-9194-96e207b3e950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:17.915811 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9956b-d52b-4e62-9194-96e207b3e950 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:17.915924 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9956b-d52b-4e62-9194-96e207b3e950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:17.916172 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9956b-d52b-4e62-9194-96e207b3e950 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:17.916920 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e9956b-d52b-4e62-9194-96e207b3e950 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:17.918610 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e9956b-d52b-4e62-9194-96e207b3e950 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:17.947735 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18f0b8-e663-42d3-b5b8-8d5d5b621bf2 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:17.951538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 860/1681] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 21:32:17.959334 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18f0b8-e663-42d3-b5b8-8d5d5b621bf2 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:17.977381 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c394bcb-750d-49ec-8643-1c9f72a0c66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:17.977381 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c394bcb-750d-49ec-8643-1c9f72a0c66b admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:17.977381 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c394bcb-750d-49ec-8643-1c9f72a0c66b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:17.977381 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c394bcb-750d-49ec-8643-1c9f72a0c66b admin admin] PATH_INFO: `/v3/projects/6402175446b748e5a4750f360e80ffa4/users/3ba531b7f6c04d44a628cc577b1fd254/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:18.024288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c394bcb-750d-49ec-8643-1c9f72a0c66b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6402175446b748e5a4750f360e80ffa4, user_id=3ba531b7f6c04d44a628cc577b1fd254, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:18.027366 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c394bcb-750d-49ec-8643-1c9f72a0c66b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:18.054509 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 822/1682] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:17 2022] PUT /identity/v3/projects/6402175446b748e5a4750f360e80ffa4/users/3ba531b7f6c04d44a628cc577b1fd254/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:32:18.064939 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234a863-074a-4011-88bb-9e891aad6d28 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:18.065422 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234a863-074a-4011-88bb-9e891aad6d28 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:18.065710 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234a863-074a-4011-88bb-9e891aad6d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:18.195214 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19179a-1de3-4c44-aa82-443ef333dad3 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:18.195565 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19179a-1de3-4c44-aa82-443ef333dad3 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:18.195979 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19179a-1de3-4c44-aa82-443ef333dad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:18.261423 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-0d19179a-1de3-4c44-aa82-443ef333dad3 None None] MFA Rules not processed for user `1589968f375946a38e82ec484afc837f`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:18.269183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0d19179a-1de3-4c44-aa82-443ef333dad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:18.363751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:18.364106 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-8234a863-074a-4011-88bb-9e891aad6d28 None None] MFA Rules not processed for user `3ba531b7f6c04d44a628cc577b1fd254`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:18.367043 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8234a863-074a-4011-88bb-9e891aad6d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:18.475854 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 823/1683] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:18.478420 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d19179a-1de3-4c44-aa82-443ef333dad3 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:18.481820 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0d19179a-1de3-4c44-aa82-443ef333dad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:18.537384 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 861/1684] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:18 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 473 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:18.538933 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a67d8-e7c7-4009-957e-3ba5475533fd None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:18.539386 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a67d8-e7c7-4009-957e-3ba5475533fd None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:18.539674 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a67d8-e7c7-4009-957e-3ba5475533fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:18.571023 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-7b4a67d8-e7c7-4009-957e-3ba5475533fd None None] MFA Rules not processed for user `1589968f375946a38e82ec484afc837f`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:18.574214 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7b4a67d8-e7c7-4009-957e-3ba5475533fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:18.588009 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 862/1685] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:18.588009 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a67d8-e7c7-4009-957e-3ba5475533fd None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:18.631979 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d19179a-1de3-4c44-aa82-443ef333dad3 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:18.642807 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec2b3bb-e59c-43ac-a3f5-39c12b66d5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:18.644152 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec2b3bb-e59c-43ac-a3f5-39c12b66d5a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:18.644911 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec2b3bb-e59c-43ac-a3f5-39c12b66d5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:18.645380 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec2b3bb-e59c-43ac-a3f5-39c12b66d5a5 admin admin] PATH_INFO: `/v3/projects/efff76d18cd741a59f46194ca09135d0` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:18.651667 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec2b3bb-e59c-43ac-a3f5-39c12b66d5a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efff76d18cd741a59f46194ca09135d0)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:18.653954 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec2b3bb-e59c-43ac-a3f5-39c12b66d5a5 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:18.715983 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a67d8-e7c7-4009-957e-3ba5475533fd None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:18.726101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b6f958-eb23-406f-9b4e-6aab4d6371c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:18.728295 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6f958-eb23-406f-9b4e-6aab4d6371c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:18.728820 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6f958-eb23-406f-9b4e-6aab4d6371c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:18.729789 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6f958-eb23-406f-9b4e-6aab4d6371c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:18.730749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b6f958-eb23-406f-9b4e-6aab4d6371c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:18.734445 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b6f958-eb23-406f-9b4e-6aab4d6371c5 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:18.774222 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0ec2b3bb-e59c-43ac-a3f5-39c12b66d5a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efff76d18cd741a59f46194ca09135d0', 'request_id': 'req-0ec2b3bb-e59c-43ac-a3f5-39c12b66d5a5'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:18.787632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0ec2b3bb-e59c-43ac-a3f5-39c12b66d5a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efff76d18cd741a59f46194ca09135d0', 'request_id': 'req-0ec2b3bb-e59c-43ac-a3f5-39c12b66d5a5'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:18.805398 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0ec2b3bb-e59c-43ac-a3f5-39c12b66d5a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efff76d18cd741a59f46194ca09135d0', 'request_id': 'req-0ec2b3bb-e59c-43ac-a3f5-39c12b66d5a5'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:18.805647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 863/1686] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:32:18 2022] POST /identity/v3/projects => generated 369 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:18.807862 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b6f958-eb23-406f-9b4e-6aab4d6371c5 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:18.812163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-84b6f958-eb23-406f-9b4e-6aab4d6371c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:18.834944 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0ec2b3bb-e59c-43ac-a3f5-39c12b66d5a5 admin admin] The token cache is being invalidate because project efff76d18cd741a59f46194ca09135d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:32:18.837162 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0ec2b3bb-e59c-43ac-a3f5-39c12b66d5a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:18.839006 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0ec2b3bb-e59c-43ac-a3f5-39c12b66d5a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:18.847046 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 824/1687] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:32:18 2022] DELETE /identity/v3/projects/efff76d18cd741a59f46194ca09135d0 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:18.847046 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec2b3bb-e59c-43ac-a3f5-39c12b66d5a5 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:18.850048 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0ec2b3bb-e59c-43ac-a3f5-39c12b66d5a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:18.968873 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:19.008237 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b6f958-eb23-406f-9b4e-6aab4d6371c5 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:19.013885 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec2b3bb-e59c-43ac-a3f5-39c12b66d5a5 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:19.032618 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:19.042595 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d46b9-8ff6-4eb0-8dc7-47f5d884e133 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:19.044631 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d46b9-8ff6-4eb0-8dc7-47f5d884e133 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:19.045826 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d46b9-8ff6-4eb0-8dc7-47f5d884e133 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:19.047412 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d46b9-8ff6-4eb0-8dc7-47f5d884e133 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:19.054892 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4f3d46b9-8ff6-4eb0-8dc7-47f5d884e133 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:19.185749 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6226af62-3810-47c3-8c94-86b8b16c5f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:19.188296 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6226af62-3810-47c3-8c94-86b8b16c5f07 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:19.191241 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6226af62-3810-47c3-8c94-86b8b16c5f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:19.192428 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6226af62-3810-47c3-8c94-86b8b16c5f07 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:19.193969 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6226af62-3810-47c3-8c94-86b8b16c5f07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:19.197737 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6226af62-3810-47c3-8c94-86b8b16c5f07 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:19.247700 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d46b9-8ff6-4eb0-8dc7-47f5d884e133 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:19.266333 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d46b9-8ff6-4eb0-8dc7-47f5d884e133 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:19.296378 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 825/1688] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:32:18 2022] POST /identity/v3/users => generated 291 bytes in 450 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:19.296378 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adce8dd-5f60-4a2a-a26d-0ddf77b5f488 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:19.296378 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adce8dd-5f60-4a2a-a26d-0ddf77b5f488 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:19.296378 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adce8dd-5f60-4a2a-a26d-0ddf77b5f488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:19.317764 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 864/1689] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:18 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:19.319981 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d46b9-8ff6-4eb0-8dc7-47f5d884e133 service nova] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:19.330785 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4f3d46b9-8ff6-4eb0-8dc7-47f5d884e133 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:19.355940 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-0adce8dd-5f60-4a2a-a26d-0ddf77b5f488 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:19.357526 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0adce8dd-5f60-4a2a-a26d-0ddf77b5f488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:19.390326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d46b9-8ff6-4eb0-8dc7-47f5d884e133 service nova] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:19.397323 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 826/1690] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:19.398675 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adce8dd-5f60-4a2a-a26d-0ddf77b5f488 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:19.402493 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67630e-efef-4782-8dd9-bff84272234f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:19.403923 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67630e-efef-4782-8dd9-bff84272234f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:19.404223 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67630e-efef-4782-8dd9-bff84272234f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:19.404511 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67630e-efef-4782-8dd9-bff84272234f admin admin] PATH_INFO: `/v3/projects/3b4b1280067b4d51ad8de574fdca512b` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:19.405871 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed67630e-efef-4782-8dd9-bff84272234f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b4b1280067b4d51ad8de574fdca512b)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:19.426690 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed67630e-efef-4782-8dd9-bff84272234f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:19.436397 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adce8dd-5f60-4a2a-a26d-0ddf77b5f488 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:19.455948 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c2fad5-449a-4b48-a7f0-98b1d094a359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:19.456663 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2fad5-449a-4b48-a7f0-98b1d094a359 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:19.456950 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2fad5-449a-4b48-a7f0-98b1d094a359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:19.457304 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2fad5-449a-4b48-a7f0-98b1d094a359 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:19.458023 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c2fad5-449a-4b48-a7f0-98b1d094a359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:19.459756 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c2fad5-449a-4b48-a7f0-98b1d094a359 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:19.543813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 827/1691] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:19.546259 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba41bf-06dc-43f9-9647-a54418f61482 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:19.546754 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba41bf-06dc-43f9-9647-a54418f61482 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:19.547214 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba41bf-06dc-43f9-9647-a54418f61482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:19.607517 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-33ba41bf-06dc-43f9-9647-a54418f61482 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:19.621235 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ed67630e-efef-4782-8dd9-bff84272234f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b4b1280067b4d51ad8de574fdca512b', 'request_id': 'req-ed67630e-efef-4782-8dd9-bff84272234f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:19.623140 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-33ba41bf-06dc-43f9-9647-a54418f61482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:19.643208 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ed67630e-efef-4782-8dd9-bff84272234f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b4b1280067b4d51ad8de574fdca512b', 'request_id': 'req-ed67630e-efef-4782-8dd9-bff84272234f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:19.656052 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ed67630e-efef-4782-8dd9-bff84272234f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b4b1280067b4d51ad8de574fdca512b', 'request_id': 'req-ed67630e-efef-4782-8dd9-bff84272234f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:19.727271 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ed67630e-efef-4782-8dd9-bff84272234f admin admin] The token cache is being invalidate because project 3b4b1280067b4d51ad8de574fdca512b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:32:19.727496 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ed67630e-efef-4782-8dd9-bff84272234f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:19.730285 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ed67630e-efef-4782-8dd9-bff84272234f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:19.734127 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 865/1692] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:32:19 2022] DELETE /identity/v3/projects/3b4b1280067b4d51ad8de574fdca512b => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:19.735259 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67630e-efef-4782-8dd9-bff84272234f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:19.739688 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ed67630e-efef-4782-8dd9-bff84272234f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:19.840198 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 828/1693] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:19.842205 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ba41bf-06dc-43f9-9647-a54418f61482 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:19.844074 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-33ba41bf-06dc-43f9-9647-a54418f61482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:19.866744 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ba41bf-06dc-43f9-9647-a54418f61482 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:19.874527 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f0cd9d-e3e0-4cd3-b4e9-9e7bb595aa34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:19.876409 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f0cd9d-e3e0-4cd3-b4e9-9e7bb595aa34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:19.877057 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f0cd9d-e3e0-4cd3-b4e9-9e7bb595aa34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:19.877469 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f0cd9d-e3e0-4cd3-b4e9-9e7bb595aa34 admin admin] PATH_INFO: `/v3/users/eb46b220318e4260bb2122cc37fc2410` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:19.879036 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f0cd9d-e3e0-4cd3-b4e9-9e7bb595aa34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb46b220318e4260bb2122cc37fc2410)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:19.882852 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f0cd9d-e3e0-4cd3-b4e9-9e7bb595aa34 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:19.924026 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67630e-efef-4782-8dd9-bff84272234f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:19.924026 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afd37eb-d653-45ba-9c6b-dcf1fae2e9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:19.924026 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afd37eb-d653-45ba-9c6b-dcf1fae2e9ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:19.924375 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afd37eb-d653-45ba-9c6b-dcf1fae2e9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:19.924734 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afd37eb-d653-45ba-9c6b-dcf1fae2e9ba admin admin] PATH_INFO: `/v3/projects/80df3b105c9c44debf9d9842ac87e58d/users/664f02bdb65a4030960f5762a82d9c1e/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:19.952184 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afd37eb-d653-45ba-9c6b-dcf1fae2e9ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=80df3b105c9c44debf9d9842ac87e58d, user_id=664f02bdb65a4030960f5762a82d9c1e, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:19.956768 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afd37eb-d653-45ba-9c6b-dcf1fae2e9ba admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:19.984352 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 866/1694] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:19 2022] PUT /identity/v3/projects/80df3b105c9c44debf9d9842ac87e58d/users/664f02bdb65a4030960f5762a82d9c1e/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:19.985466 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afd37eb-d653-45ba-9c6b-dcf1fae2e9ba admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:20.006101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:20.012948 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-01f0cd9d-e3e0-4cd3-b4e9-9e7bb595aa34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb46b220318e4260bb2122cc37fc2410'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:20.028290 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-01f0cd9d-e3e0-4cd3-b4e9-9e7bb595aa34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb46b220318e4260bb2122cc37fc2410'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:20.081205 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-01f0cd9d-e3e0-4cd3-b4e9-9e7bb595aa34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb46b220318e4260bb2122cc37fc2410'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:20.095225 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-01f0cd9d-e3e0-4cd3-b4e9-9e7bb595aa34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb46b220318e4260bb2122cc37fc2410'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:20.098565 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:20.116929 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-01f0cd9d-e3e0-4cd3-b4e9-9e7bb595aa34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb46b220318e4260bb2122cc37fc2410'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:20.129447 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 829/1695] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:32:19 2022] DELETE /identity/v3/users/eb46b220318e4260bb2122cc37fc2410 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:20.132132 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f0cd9d-e3e0-4cd3-b4e9-9e7bb595aa34 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:20.134151 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-01f0cd9d-e3e0-4cd3-b4e9-9e7bb595aa34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:20.207259 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afd37eb-d653-45ba-9c6b-dcf1fae2e9ba admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:20.338996 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f0cd9d-e3e0-4cd3-b4e9-9e7bb595aa34 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:20.355322 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b3aa5-7f8e-4f05-81fa-acf1320e59f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:20.356366 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b3aa5-7f8e-4f05-81fa-acf1320e59f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:20.357449 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b3aa5-7f8e-4f05-81fa-acf1320e59f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:20.359495 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b3aa5-7f8e-4f05-81fa-acf1320e59f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:20.366095 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312b3aa5-7f8e-4f05-81fa-acf1320e59f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:20.374523 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312b3aa5-7f8e-4f05-81fa-acf1320e59f6 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:20.378001 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470dfc50-0a8c-43f6-820c-c620986d4842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:20.379128 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470dfc50-0a8c-43f6-820c-c620986d4842 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:20.379556 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470dfc50-0a8c-43f6-820c-c620986d4842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:20.380530 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470dfc50-0a8c-43f6-820c-c620986d4842 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:20.381794 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470dfc50-0a8c-43f6-820c-c620986d4842 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:32:20.382848 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470dfc50-0a8c-43f6-820c-c620986d4842 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:20.396877 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470dfc50-0a8c-43f6-820c-c620986d4842 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:20.412380 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 830/1696] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:20.418494 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b3aa5-7f8e-4f05-81fa-acf1320e59f6 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:20.435598 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 867/1697] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:32:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 451 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:32:20.440527 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470dfc50-0a8c-43f6-820c-c620986d4842 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:20.448132 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b3aa5-7f8e-4f05-81fa-acf1320e59f6 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:20.452104 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-470dfc50-0a8c-43f6-820c-c620986d4842 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:20.466716 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa53da8-b155-497a-9e58-bfff94ff5dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:20.467770 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa53da8-b155-497a-9e58-bfff94ff5dc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:20.468108 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa53da8-b155-497a-9e58-bfff94ff5dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:20.469118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa53da8-b155-497a-9e58-bfff94ff5dc0 admin admin] PATH_INFO: `/v3/projects/80df3b105c9c44debf9d9842ac87e58d/users/664f02bdb65a4030960f5762a82d9c1e/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:20.476001 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa53da8-b155-497a-9e58-bfff94ff5dc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80df3b105c9c44debf9d9842ac87e58d, user_id=664f02bdb65a4030960f5762a82d9c1e, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:20.477268 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470dfc50-0a8c-43f6-820c-c620986d4842 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:20.483273 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa53da8-b155-497a-9e58-bfff94ff5dc0 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:20.493838 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf60a1c-fc33-48ed-ae0b-683c7e2150e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:20.495048 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf60a1c-fc33-48ed-ae0b-683c7e2150e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:20.495438 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf60a1c-fc33-48ed-ae0b-683c7e2150e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:20.495822 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf60a1c-fc33-48ed-ae0b-683c7e2150e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:20.496701 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf60a1c-fc33-48ed-ae0b-683c7e2150e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:20.502360 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf60a1c-fc33-48ed-ae0b-683c7e2150e9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:20.509894 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 831/1698] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:20 2022] PUT /identity/v3/projects/80df3b105c9c44debf9d9842ac87e58d/users/664f02bdb65a4030960f5762a82d9c1e/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:20.522777 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84232b-51e1-41e6-b005-c2498183b159 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:20.523154 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84232b-51e1-41e6-b005-c2498183b159 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:20.523350 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84232b-51e1-41e6-b005-c2498183b159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:20.618705 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 868/1699] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:32:20 2022] POST /identity/v3/projects => generated 399 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:20.623388 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf60a1c-fc33-48ed-ae0b-683c7e2150e9 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:20.638183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-9a84232b-51e1-41e6-b005-c2498183b159 None None] MFA Rules not processed for user `664f02bdb65a4030960f5762a82d9c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:20.639771 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9a84232b-51e1-41e6-b005-c2498183b159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:20.767472 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf60a1c-fc33-48ed-ae0b-683c7e2150e9 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:20.773560 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c249a25b-ad9c-4d7a-9c5b-7005cc3bed90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:20.774438 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c249a25b-ad9c-4d7a-9c5b-7005cc3bed90 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:20.774791 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c249a25b-ad9c-4d7a-9c5b-7005cc3bed90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:20.775195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c249a25b-ad9c-4d7a-9c5b-7005cc3bed90 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:20.776240 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c249a25b-ad9c-4d7a-9c5b-7005cc3bed90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:20.779300 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c249a25b-ad9c-4d7a-9c5b-7005cc3bed90 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:20.814304 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 869/1700] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:32:20 2022] POST /identity/v3/users => generated 300 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:20.817734 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83aec87-d226-487b-baf8-30cc0da17ad1 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:20.819705 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83aec87-d226-487b-baf8-30cc0da17ad1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:20.820534 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83aec87-d226-487b-baf8-30cc0da17ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:20.857499 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 832/1701] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:20 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:20.858382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84232b-51e1-41e6-b005-c2498183b159 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:20.877911 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a83aec87-d226-487b-baf8-30cc0da17ad1 None None] MFA Rules not processed for user `eba91dff7cac401e81f6fb6391874af1`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:20.880600 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a83aec87-d226-487b-baf8-30cc0da17ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:20.887591 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84232b-51e1-41e6-b005-c2498183b159 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:20.895781 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:20.902271 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5645db86-40dc-41ea-a1f8-482e57a5312f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:20.903059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645db86-40dc-41ea-a1f8-482e57a5312f admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:20.903551 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645db86-40dc-41ea-a1f8-482e57a5312f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:20.903657 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645db86-40dc-41ea-a1f8-482e57a5312f admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:20.904432 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5645db86-40dc-41ea-a1f8-482e57a5312f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:20.910911 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5645db86-40dc-41ea-a1f8-482e57a5312f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:20.942908 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 833/1702] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:20.944495 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5645db86-40dc-41ea-a1f8-482e57a5312f admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:20.949442 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5645db86-40dc-41ea-a1f8-482e57a5312f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:21.022400 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 870/1703] 10.222.0.65 () {58 vars in 1009 bytes} [Thu May 19 21:32:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:21.024146 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83aec87-d226-487b-baf8-30cc0da17ad1 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:21.061312 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83aec87-d226-487b-baf8-30cc0da17ad1 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:21.072113 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33d161d-6a66-450a-9db9-6ca6187a7ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:21.072788 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d161d-6a66-450a-9db9-6ca6187a7ae4 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:21.073074 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d161d-6a66-450a-9db9-6ca6187a7ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:21.073347 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d161d-6a66-450a-9db9-6ca6187a7ae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:21.074088 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33d161d-6a66-450a-9db9-6ca6187a7ae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:21.075723 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33d161d-6a66-450a-9db9-6ca6187a7ae4 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:21.084802 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:INSERT=2,keystone:DELETE=1,keystone:SELECT=97 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:21.094497 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5645db86-40dc-41ea-a1f8-482e57a5312f admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:21.103087 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc84fb-255b-4026-b5ec-9539c181106b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:21.104195 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc84fb-255b-4026-b5ec-9539c181106b service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:21.104543 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc84fb-255b-4026-b5ec-9539c181106b service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:21.104925 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc84fb-255b-4026-b5ec-9539c181106b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:21.112724 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 871/1704] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:21.115575 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d2513-dc7b-4c2b-a088-bb9cbe3f5520 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:21.122148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d2513-dc7b-4c2b-a088-bb9cbe3f5520 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:21.122680 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d2513-dc7b-4c2b-a088-bb9cbe3f5520 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:21.131700 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfc84fb-255b-4026-b5ec-9539c181106b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:21.132373 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 872/1705] 10.222.0.65 () {54 vars in 912 bytes} [Thu May 19 21:32:21 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:32:21.133505 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d2513-dc7b-4c2b-a088-bb9cbe3f5520 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:21.134427 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfc84fb-255b-4026-b5ec-9539c181106b service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:21.156215 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 834/1706] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:20 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:21.159886 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ad7db-c14b-46fb-a6b9-35823d1aa8c9 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:21.162422 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ad7db-c14b-46fb-a6b9-35823d1aa8c9 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:21.164665 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ad7db-c14b-46fb-a6b9-35823d1aa8c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:21.171393 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d2513-dc7b-4c2b-a088-bb9cbe3f5520 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:21.191119 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d4d039-7e02-4d6c-acdd-cb46fd825815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:21.192639 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4d039-7e02-4d6c-acdd-cb46fd825815 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:21.193118 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4d039-7e02-4d6c-acdd-cb46fd825815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:21.193638 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4d039-7e02-4d6c-acdd-cb46fd825815 admin admin] PATH_INFO: `/v3/projects/0ceaaceb6bce449aa1ccd51de3a605ee/users/d72dd5014aa94f03a9ebfaaeea053710/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:21.218572 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-a97ad7db-c14b-46fb-a6b9-35823d1aa8c9 None None] MFA Rules not processed for user `2201e66caa4b4a57be09d845d7cb0023`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:21.235775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a97ad7db-c14b-46fb-a6b9-35823d1aa8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:21.247070 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d4d039-7e02-4d6c-acdd-cb46fd825815 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ceaaceb6bce449aa1ccd51de3a605ee, user_id=d72dd5014aa94f03a9ebfaaeea053710, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:21.256297 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d4d039-7e02-4d6c-acdd-cb46fd825815 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:21.317092 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 873/1707] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:21 2022] PUT /identity/v3/projects/0ceaaceb6bce449aa1ccd51de3a605ee/users/d72dd5014aa94f03a9ebfaaeea053710/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:21.332285 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d4d039-7e02-4d6c-acdd-cb46fd825815 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:21.340030 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-51d4d039-7e02-4d6c-acdd-cb46fd825815 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:21.418162 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 835/1708] 10.222.0.65 () {58 vars in 1009 bytes} [Thu May 19 21:32:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:21.419848 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82d09b-dbc5-4935-8812-914bd2e89ded None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:21.422295 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82d09b-dbc5-4935-8812-914bd2e89ded None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:21.423447 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82d09b-dbc5-4935-8812-914bd2e89ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:21.510165 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-ac82d09b-dbc5-4935-8812-914bd2e89ded None None] MFA Rules not processed for user `d72dd5014aa94f03a9ebfaaeea053710`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:21.514679 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ac82d09b-dbc5-4935-8812-914bd2e89ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:21.632861 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d4d039-7e02-4d6c-acdd-cb46fd825815 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:21.661157 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd508851-30f9-471c-bebd-a7dda404e240 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e07984b36934aff9052541ef2a56f36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:21.661858 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd508851-30f9-471c-bebd-a7dda404e240 service swift] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:21.662145 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd508851-30f9-471c-bebd-a7dda404e240 service swift] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:21.670589 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd508851-30f9-471c-bebd-a7dda404e240 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:21.680428 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 836/1709] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:21 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:21.684174 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac82d09b-dbc5-4935-8812-914bd2e89ded None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:21.686061 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ac82d09b-dbc5-4935-8812-914bd2e89ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:21.834136 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac82d09b-dbc5-4935-8812-914bd2e89ded None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:21.844828 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c6260-90d2-4eac-b1b0-2127ab8e8940 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:21.845520 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c6260-90d2-4eac-b1b0-2127ab8e8940 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:21.845974 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c6260-90d2-4eac-b1b0-2127ab8e8940 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:21.846282 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c6260-90d2-4eac-b1b0-2127ab8e8940 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:21.869569 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd508851-30f9-471c-bebd-a7dda404e240 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:21.872298 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd508851-30f9-471c-bebd-a7dda404e240 service swift] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:21.872528 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4c6260-90d2-4eac-b1b0-2127ab8e8940 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:21.874014 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4c6260-90d2-4eac-b1b0-2127ab8e8940 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:21.891334 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 837/1710] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:21.892247 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c6260-90d2-4eac-b1b0-2127ab8e8940 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:21.915903 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 874/1711] 10.222.0.65 () {60 vars in 1384 bytes} [Thu May 19 21:32:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 21:32:21.978801 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:22.043049 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c6260-90d2-4eac-b1b0-2127ab8e8940 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:22.058203 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1dd18-57e4-41fe-a818-60ee3f06697a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:22.058924 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1dd18-57e4-41fe-a818-60ee3f06697a admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:22.059214 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1dd18-57e4-41fe-a818-60ee3f06697a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:22.059520 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1dd18-57e4-41fe-a818-60ee3f06697a admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:22.060997 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f1dd18-57e4-41fe-a818-60ee3f06697a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:22.063099 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f1dd18-57e4-41fe-a818-60ee3f06697a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:22.127507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 838/1712] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:32:21 2022] POST /identity/v3/projects => generated 399 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:22.131938 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd508851-30f9-471c-bebd-a7dda404e240 service swift] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:22.155907 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd508851-30f9-471c-bebd-a7dda404e240 service swift] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:22.312531 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:22.532899 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae53053e-3795-4add-8ed9-f6f018c74c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:22.533540 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae53053e-3795-4add-8ed9-f6f018c74c7c admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:22.533826 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae53053e-3795-4add-8ed9-f6f018c74c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:22.534128 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae53053e-3795-4add-8ed9-f6f018c74c7c admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:22.536227 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae53053e-3795-4add-8ed9-f6f018c74c7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:22.545092 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae53053e-3795-4add-8ed9-f6f018c74c7c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:22.629938 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 875/1713] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:32:22 2022] POST /identity/v3/users => generated 306 bytes in 498 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:22.634209 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1dd18-57e4-41fe-a818-60ee3f06697a admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:22.664988 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1dd18-57e4-41fe-a818-60ee3f06697a admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:22.702584 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022b7c13-4abc-48b4-bc64-8481cb7bea58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:22.706912 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b7c13-4abc-48b4-bc64-8481cb7bea58 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:22.707370 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b7c13-4abc-48b4-bc64-8481cb7bea58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:22.707735 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b7c13-4abc-48b4-bc64-8481cb7bea58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:22.710637 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022b7c13-4abc-48b4-bc64-8481cb7bea58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:22.712442 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022b7c13-4abc-48b4-bc64-8481cb7bea58 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:22.748061 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 839/1714] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:22.751134 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae53053e-3795-4add-8ed9-f6f018c74c7c admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:22.767641 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae53053e-3795-4add-8ed9-f6f018c74c7c admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:22.793198 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703be0f6-dd9a-4519-a83c-b64200f66e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:22.807830 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703be0f6-dd9a-4519-a83c-b64200f66e49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:22.808222 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703be0f6-dd9a-4519-a83c-b64200f66e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:22.808605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703be0f6-dd9a-4519-a83c-b64200f66e49 admin admin] PATH_INFO: `/v3/projects/fddb2c9bfa0e4d9c9cd2bdf2b28b1616/users/681894709e614c918144b6b405c23ee5/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:22.860870 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703be0f6-dd9a-4519-a83c-b64200f66e49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fddb2c9bfa0e4d9c9cd2bdf2b28b1616, user_id=681894709e614c918144b6b405c23ee5, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:22.872033 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703be0f6-dd9a-4519-a83c-b64200f66e49 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:22.945723 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 876/1715] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:22 2022] PUT /identity/v3/projects/fddb2c9bfa0e4d9c9cd2bdf2b28b1616/users/681894709e614c918144b6b405c23ee5/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:22.951408 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022b7c13-4abc-48b4-bc64-8481cb7bea58 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:23.019781 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022b7c13-4abc-48b4-bc64-8481cb7bea58 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:23.356320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a434e960-78b8-4c94-9b68-d5211d15ed71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:23.358338 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434e960-78b8-4c94-9b68-d5211d15ed71 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:23.359024 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434e960-78b8-4c94-9b68-d5211d15ed71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:23.359475 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434e960-78b8-4c94-9b68-d5211d15ed71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:23.382150 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a434e960-78b8-4c94-9b68-d5211d15ed71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:23.412659 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a434e960-78b8-4c94-9b68-d5211d15ed71 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:23.479570 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703be0f6-dd9a-4519-a83c-b64200f66e49 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:23.494945 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 840/1716] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 545 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:23.498096 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703be0f6-dd9a-4519-a83c-b64200f66e49 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:23.505304 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd054cc-9957-45e6-9c5b-963ff5bfbc09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:23.506597 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd054cc-9957-45e6-9c5b-963ff5bfbc09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:23.506888 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd054cc-9957-45e6-9c5b-963ff5bfbc09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:23.507167 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd054cc-9957-45e6-9c5b-963ff5bfbc09 admin admin] PATH_INFO: `/v3/projects/fddb2c9bfa0e4d9c9cd2bdf2b28b1616/users/681894709e614c918144b6b405c23ee5/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:23.509775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd054cc-9957-45e6-9c5b-963ff5bfbc09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fddb2c9bfa0e4d9c9cd2bdf2b28b1616, user_id=681894709e614c918144b6b405c23ee5, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:23.514638 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd054cc-9957-45e6-9c5b-963ff5bfbc09 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:23.582493 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 877/1717] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:23 2022] PUT /identity/v3/projects/fddb2c9bfa0e4d9c9cd2bdf2b28b1616/users/681894709e614c918144b6b405c23ee5/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:23.589211 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b24a0-5822-417c-8aac-33bfdbba74aa None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:23.589777 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b24a0-5822-417c-8aac-33bfdbba74aa None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:23.590474 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b24a0-5822-417c-8aac-33bfdbba74aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:23.691453 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169751eb-6875-4b0a-8aa2-8713373decef None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:23.691835 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169751eb-6875-4b0a-8aa2-8713373decef None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:23.692123 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169751eb-6875-4b0a-8aa2-8713373decef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:23.731980 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-d11b24a0-5822-417c-8aac-33bfdbba74aa None None] MFA Rules not processed for user `681894709e614c918144b6b405c23ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:23.733816 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d11b24a0-5822-417c-8aac-33bfdbba74aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:23.808605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-169751eb-6875-4b0a-8aa2-8713373decef None None] MFA Rules not processed for user `2201e66caa4b4a57be09d845d7cb0023`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:23.810081 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-169751eb-6875-4b0a-8aa2-8713373decef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:24.193737 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 878/1718] 10.222.0.87 () {58 vars in 1015 bytes} [Thu May 19 21:32:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 504 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:24.196724 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169751eb-6875-4b0a-8aa2-8713373decef None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:24.214117 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-169751eb-6875-4b0a-8aa2-8713373decef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:24.292882 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 841/1719] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:23 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 704 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:24.295041 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaccb2de-aec1-4265-bdd5-eefeea5b87db None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:24.295334 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaccb2de-aec1-4265-bdd5-eefeea5b87db None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:24.295611 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaccb2de-aec1-4265-bdd5-eefeea5b87db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:24.502454 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-aaccb2de-aec1-4265-bdd5-eefeea5b87db None None] MFA Rules not processed for user `f47e7f746a014cb0a35390e7e0ce435d`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:24.511316 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-aaccb2de-aec1-4265-bdd5-eefeea5b87db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:24.627152 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:24.631721 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169751eb-6875-4b0a-8aa2-8713373decef None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:24.664524 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7009462b-6afc-46ec-aed6-87a73c6061e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:24.667658 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7009462b-6afc-46ec-aed6-87a73c6061e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:24.685073 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7009462b-6afc-46ec-aed6-87a73c6061e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:24.685578 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7009462b-6afc-46ec-aed6-87a73c6061e6 admin admin] PATH_INFO: `/v3/users/04d032c7e7fb4eaaa9fdf3e92d7e28b8` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:24.691388 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7009462b-6afc-46ec-aed6-87a73c6061e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04d032c7e7fb4eaaa9fdf3e92d7e28b8)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:24.696961 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7009462b-6afc-46ec-aed6-87a73c6061e6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:24.767302 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:24.814117 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7009462b-6afc-46ec-aed6-87a73c6061e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04d032c7e7fb4eaaa9fdf3e92d7e28b8'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:24.898161 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7009462b-6afc-46ec-aed6-87a73c6061e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04d032c7e7fb4eaaa9fdf3e92d7e28b8'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:24.899266 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7009462b-6afc-46ec-aed6-87a73c6061e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04d032c7e7fb4eaaa9fdf3e92d7e28b8'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:24.900125 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7009462b-6afc-46ec-aed6-87a73c6061e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04d032c7e7fb4eaaa9fdf3e92d7e28b8'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:24.937193 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7009462b-6afc-46ec-aed6-87a73c6061e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04d032c7e7fb4eaaa9fdf3e92d7e28b8'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:25.005713 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 842/1720] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 712 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:25.007229 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaccb2de-aec1-4265-bdd5-eefeea5b87db None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:25.007909 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 879/1721] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:32:24 2022] DELETE /identity/v3/users/04d032c7e7fb4eaaa9fdf3e92d7e28b8 => generated 0 bytes in 813 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:25.009524 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7009462b-6afc-46ec-aed6-87a73c6061e6 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:25.011492 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-aaccb2de-aec1-4265-bdd5-eefeea5b87db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:25.016487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7009462b-6afc-46ec-aed6-87a73c6061e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:25.056840 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaccb2de-aec1-4265-bdd5-eefeea5b87db None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:25.337471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7009462b-6afc-46ec-aed6-87a73c6061e6 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:25.382748 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39358c0a-4308-4c49-93c3-99d54dc7e2a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:25.389821 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39358c0a-4308-4c49-93c3-99d54dc7e2a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:25.390324 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39358c0a-4308-4c49-93c3-99d54dc7e2a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:25.394807 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39358c0a-4308-4c49-93c3-99d54dc7e2a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:25.413328 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-39358c0a-4308-4c49-93c3-99d54dc7e2a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:25.447887 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b00be3-b69f-4c30-92a5-906bd141f24a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:25.449516 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b00be3-b69f-4c30-92a5-906bd141f24a service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:25.449874 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b00be3-b69f-4c30-92a5-906bd141f24a service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:25.450178 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b00be3-b69f-4c30-92a5-906bd141f24a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:25.457656 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f8b00be3-b69f-4c30-92a5-906bd141f24a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:25.490316 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b00be3-b69f-4c30-92a5-906bd141f24a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:25.492012 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b00be3-b69f-4c30-92a5-906bd141f24a service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:25.527954 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 843/1722] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:25.529433 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ee1a8-7748-4d1a-95c3-5910424f4367 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:25.532221 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ee1a8-7748-4d1a-95c3-5910424f4367 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:25.532451 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ee1a8-7748-4d1a-95c3-5910424f4367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:25.637113 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-7a3ee1a8-7748-4d1a-95c3-5910424f4367 None None] MFA Rules not processed for user `1589968f375946a38e82ec484afc837f`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:25.638407 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7a3ee1a8-7748-4d1a-95c3-5910424f4367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:25.725012 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39358c0a-4308-4c49-93c3-99d54dc7e2a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:25.730702 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39358c0a-4308-4c49-93c3-99d54dc7e2a7 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:25.796834 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 880/1723] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:25 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 788 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:25.797644 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39358c0a-4308-4c49-93c3-99d54dc7e2a7 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:25.803231 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-39358c0a-4308-4c49-93c3-99d54dc7e2a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:25.938857 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 844/1724] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:25.943805 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ee1a8-7748-4d1a-95c3-5910424f4367 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:25.947781 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7a3ee1a8-7748-4d1a-95c3-5910424f4367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:26.091932 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:26.121508 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39358c0a-4308-4c49-93c3-99d54dc7e2a7 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:26.147428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f044f-1356-4e4e-93aa-fb22d2fca7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:26.148424 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f044f-1356-4e4e-93aa-fb22d2fca7e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:26.148711 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f044f-1356-4e4e-93aa-fb22d2fca7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:26.148991 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f044f-1356-4e4e-93aa-fb22d2fca7e2 admin admin] PATH_INFO: `/v3/projects/fc1127658625452993e72e10f80b90a2` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:26.151834 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24f044f-1356-4e4e-93aa-fb22d2fca7e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc1127658625452993e72e10f80b90a2)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:26.157853 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24f044f-1356-4e4e-93aa-fb22d2fca7e2 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:26.264703 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:26.276838 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ee1a8-7748-4d1a-95c3-5910424f4367 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:26.298604 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d661bc9-78b0-4a12-a1b6-18887877b8fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:26.299283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d661bc9-78b0-4a12-a1b6-18887877b8fb service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:26.299583 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d661bc9-78b0-4a12-a1b6-18887877b8fb service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:26.299843 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d661bc9-78b0-4a12-a1b6-18887877b8fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:26.308297 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5d661bc9-78b0-4a12-a1b6-18887877b8fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:26.435125 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e24f044f-1356-4e4e-93aa-fb22d2fca7e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc1127658625452993e72e10f80b90a2', 'request_id': 'req-e24f044f-1356-4e4e-93aa-fb22d2fca7e2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:26.459113 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e24f044f-1356-4e4e-93aa-fb22d2fca7e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc1127658625452993e72e10f80b90a2', 'request_id': 'req-e24f044f-1356-4e4e-93aa-fb22d2fca7e2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:26.492307 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e24f044f-1356-4e4e-93aa-fb22d2fca7e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc1127658625452993e72e10f80b90a2', 'request_id': 'req-e24f044f-1356-4e4e-93aa-fb22d2fca7e2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:26.518782 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d661bc9-78b0-4a12-a1b6-18887877b8fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:26.521480 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d661bc9-78b0-4a12-a1b6-18887877b8fb service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:26.532891 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e24f044f-1356-4e4e-93aa-fb22d2fca7e2 admin admin] The token cache is being invalidate because project fc1127658625452993e72e10f80b90a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:32:26.533435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e24f044f-1356-4e4e-93aa-fb22d2fca7e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:26.548437 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e24f044f-1356-4e4e-93aa-fb22d2fca7e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:26.559729 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 881/1725] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:32:25 2022] DELETE /identity/v3/projects/fc1127658625452993e72e10f80b90a2 => generated 0 bytes in 758 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:26.561605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9e364-7ecb-43ad-b650-a9b4c91dd54d None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:26.562937 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9e364-7ecb-43ad-b650-a9b4c91dd54d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:26.563677 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9e364-7ecb-43ad-b650-a9b4c91dd54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:26.580292 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 845/1726] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:25 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 641 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:26.582490 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d661bc9-78b0-4a12-a1b6-18887877b8fb service nova] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:26.614388 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d661bc9-78b0-4a12-a1b6-18887877b8fb service nova] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:26.669844 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c6f9e364-7ecb-43ad-b650-a9b4c91dd54d None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:26.676267 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c6f9e364-7ecb-43ad-b650-a9b4c91dd54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:26.803629 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc73a805-9307-4998-8611-e78046bced99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:26.804562 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc73a805-9307-4998-8611-e78046bced99 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:26.806613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc73a805-9307-4998-8611-e78046bced99 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:26.806921 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc73a805-9307-4998-8611-e78046bced99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:26.809989 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-dc73a805-9307-4998-8611-e78046bced99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:26.931615 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 882/1727] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:26.932673 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9e364-7ecb-43ad-b650-a9b4c91dd54d None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:26.936598 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c6f9e364-7ecb-43ad-b650-a9b4c91dd54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:26.964053 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9e364-7ecb-43ad-b650-a9b4c91dd54d None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:26.970922 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6bfe5e-3f2e-4080-8763-7b4da6000ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:26.972598 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6bfe5e-3f2e-4080-8763-7b4da6000ad4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:26.973304 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6bfe5e-3f2e-4080-8763-7b4da6000ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:26.973999 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6bfe5e-3f2e-4080-8763-7b4da6000ad4 admin admin] PATH_INFO: `/v3/users/3ba531b7f6c04d44a628cc577b1fd254` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:26.975963 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6bfe5e-3f2e-4080-8763-7b4da6000ad4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ba531b7f6c04d44a628cc577b1fd254)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:26.978537 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6bfe5e-3f2e-4080-8763-7b4da6000ad4 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:27.003137 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc73a805-9307-4998-8611-e78046bced99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:27.007532 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc73a805-9307-4998-8611-e78046bced99 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:27.023701 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 846/1728] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:26 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:27.025420 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab0be8-e323-4c99-b2b3-37e499b8b071 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:27.025870 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab0be8-e323-4c99-b2b3-37e499b8b071 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:27.026485 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab0be8-e323-4c99-b2b3-37e499b8b071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:27.060021 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-a7ab0be8-e323-4c99-b2b3-37e499b8b071 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:27.061853 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a7ab0be8-e323-4c99-b2b3-37e499b8b071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:27.079760 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 847/1729] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:27.083387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab0be8-e323-4c99-b2b3-37e499b8b071 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:27.091187 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a7ab0be8-e323-4c99-b2b3-37e499b8b071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:27.117329 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0b6bfe5e-3f2e-4080-8763-7b4da6000ad4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ba531b7f6c04d44a628cc577b1fd254'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:27.149435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0b6bfe5e-3f2e-4080-8763-7b4da6000ad4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ba531b7f6c04d44a628cc577b1fd254'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:27.151952 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0b6bfe5e-3f2e-4080-8763-7b4da6000ad4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ba531b7f6c04d44a628cc577b1fd254'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:27.153087 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0b6bfe5e-3f2e-4080-8763-7b4da6000ad4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ba531b7f6c04d44a628cc577b1fd254'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:27.201171 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0b6bfe5e-3f2e-4080-8763-7b4da6000ad4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ba531b7f6c04d44a628cc577b1fd254'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:27.227416 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:27.241175 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 883/1730] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:32:26 2022] DELETE /identity/v3/users/3ba531b7f6c04d44a628cc577b1fd254 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:27.243887 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d0d37-20a9-4e8f-becd-979a1a0a8530 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:27.244184 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d0d37-20a9-4e8f-becd-979a1a0a8530 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:27.244467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d0d37-20a9-4e8f-becd-979a1a0a8530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:27.350723 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-cf2d0d37-20a9-4e8f-becd-979a1a0a8530 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:27.355102 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cf2d0d37-20a9-4e8f-becd-979a1a0a8530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:27.480840 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab0be8-e323-4c99-b2b3-37e499b8b071 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:27.492293 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:27.661178 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 884/1731] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:27.675193 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7807c6-90be-4939-b891-be212fc47fb5 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:27.676217 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7807c6-90be-4939-b891-be212fc47fb5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:27.676420 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7807c6-90be-4939-b891-be212fc47fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:27.735374 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85589474-018d-4902-ad47-fda5c5dfde3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:27.736280 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85589474-018d-4902-ad47-fda5c5dfde3e service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:27.736280 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85589474-018d-4902-ad47-fda5c5dfde3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:27.736568 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85589474-018d-4902-ad47-fda5c5dfde3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:27.763394 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-85589474-018d-4902-ad47-fda5c5dfde3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:27.778740 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-2d7807c6-90be-4939-b891-be212fc47fb5 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:27.780026 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2d7807c6-90be-4939-b891-be212fc47fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:27.827027 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85589474-018d-4902-ad47-fda5c5dfde3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:27.829149 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85589474-018d-4902-ad47-fda5c5dfde3e service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:27.835143 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 885/1732] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:27.836151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7807c6-90be-4939-b891-be212fc47fb5 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:27.856189 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7807c6-90be-4939-b891-be212fc47fb5 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:27.876627 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 848/1733] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 782 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:27.894491 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc93a446-0103-44da-a621-a410ea9d2603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:27.895859 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93a446-0103-44da-a621-a410ea9d2603 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:27.896639 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93a446-0103-44da-a621-a410ea9d2603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:27.897178 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93a446-0103-44da-a621-a410ea9d2603 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:27.898094 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc93a446-0103-44da-a621-a410ea9d2603 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:32:27.898587 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc93a446-0103-44da-a621-a410ea9d2603 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:27.903646 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc93a446-0103-44da-a621-a410ea9d2603 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:27.933054 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 886/1734] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:32:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:32:27.934597 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85589474-018d-4902-ad47-fda5c5dfde3e service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:27.946484 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85589474-018d-4902-ad47-fda5c5dfde3e service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:27.953698 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50453226-7d64-4f41-b61a-dab7f1875d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:27.954921 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50453226-7d64-4f41-b61a-dab7f1875d20 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:27.955579 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50453226-7d64-4f41-b61a-dab7f1875d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:27.956029 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50453226-7d64-4f41-b61a-dab7f1875d20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:27.958631 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc93a446-0103-44da-a621-a410ea9d2603 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:27.960847 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fc93a446-0103-44da-a621-a410ea9d2603 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:27.963461 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50453226-7d64-4f41-b61a-dab7f1875d20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:27.976262 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50453226-7d64-4f41-b61a-dab7f1875d20 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:27.987465 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc93a446-0103-44da-a621-a410ea9d2603 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:28.002211 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf39c50d-e81d-47df-b543-9758f5f3eb99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:28.002510 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39c50d-e81d-47df-b543-9758f5f3eb99 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:28.002650 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39c50d-e81d-47df-b543-9758f5f3eb99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:28.003087 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39c50d-e81d-47df-b543-9758f5f3eb99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:28.031684 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf39c50d-e81d-47df-b543-9758f5f3eb99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:28.036778 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf39c50d-e81d-47df-b543-9758f5f3eb99 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:28.050324 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 849/1735] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:32:27 2022] POST /identity/v3/projects => generated 379 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:28.051876 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50453226-7d64-4f41-b61a-dab7f1875d20 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:28.082657 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 887/1736] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:28.217450 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50453226-7d64-4f41-b61a-dab7f1875d20 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:28.234990 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b924a7a-adf8-4545-bba9-b0c46e7c0870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:28.235975 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b924a7a-adf8-4545-bba9-b0c46e7c0870 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:28.235975 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b924a7a-adf8-4545-bba9-b0c46e7c0870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:28.236288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b924a7a-adf8-4545-bba9-b0c46e7c0870 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:28.237152 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b924a7a-adf8-4545-bba9-b0c46e7c0870 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:28.244807 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b924a7a-adf8-4545-bba9-b0c46e7c0870 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:28.310758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf39c50d-e81d-47df-b543-9758f5f3eb99 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:28.324495 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 850/1737] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:32:28 2022] POST /identity/v3/users => generated 290 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:28.332908 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf39c50d-e81d-47df-b543-9758f5f3eb99 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:28.349290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec8e543-1b57-4aa0-8d37-a98457939cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:28.351792 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec8e543-1b57-4aa0-8d37-a98457939cc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:28.352900 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec8e543-1b57-4aa0-8d37-a98457939cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:28.353978 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec8e543-1b57-4aa0-8d37-a98457939cc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:28.356280 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec8e543-1b57-4aa0-8d37-a98457939cc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:28.359831 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec8e543-1b57-4aa0-8d37-a98457939cc0 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:28.387159 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 888/1738] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 21:32:28.391765 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b924a7a-adf8-4545-bba9-b0c46e7c0870 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:28.408378 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b924a7a-adf8-4545-bba9-b0c46e7c0870 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:28.423504 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fe1a01-6a0b-4fe7-a0e3-86524331e59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:28.424583 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe1a01-6a0b-4fe7-a0e3-86524331e59c admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:28.424998 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe1a01-6a0b-4fe7-a0e3-86524331e59c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:28.426484 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe1a01-6a0b-4fe7-a0e3-86524331e59c admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:28.427473 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fe1a01-6a0b-4fe7-a0e3-86524331e59c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:28.430376 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fe1a01-6a0b-4fe7-a0e3-86524331e59c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:28.461370 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec8e543-1b57-4aa0-8d37-a98457939cc0 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:28.476404 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec8e543-1b57-4aa0-8d37-a98457939cc0 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:28.478874 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 851/1739] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:28.488889 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06fba9-458c-4a1e-b758-17fcad7f433c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:28.489968 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06fba9-458c-4a1e-b758-17fcad7f433c admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:28.490273 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06fba9-458c-4a1e-b758-17fcad7f433c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:28.490789 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06fba9-458c-4a1e-b758-17fcad7f433c admin admin] PATH_INFO: `/v3/projects/e0082fc7723648ccb2ba89573769debf/users/4377ac3e0932476e88bc9c91e5475c3d/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:28.590430 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c06fba9-458c-4a1e-b758-17fcad7f433c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0082fc7723648ccb2ba89573769debf, user_id=4377ac3e0932476e88bc9c91e5475c3d, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:28.604579 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c06fba9-458c-4a1e-b758-17fcad7f433c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:28.665454 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 889/1740] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:28 2022] PUT /identity/v3/projects/e0082fc7723648ccb2ba89573769debf/users/4377ac3e0932476e88bc9c91e5475c3d/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:32:28.677144 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a72bde-9dec-46d2-ab9c-aea3b92cea4a None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:28.677517 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a72bde-9dec-46d2-ab9c-aea3b92cea4a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:28.678033 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a72bde-9dec-46d2-ab9c-aea3b92cea4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:28.776964 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-f4a72bde-9dec-46d2-ab9c-aea3b92cea4a None None] MFA Rules not processed for user `4377ac3e0932476e88bc9c91e5475c3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:28.779539 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f4a72bde-9dec-46d2-ab9c-aea3b92cea4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:28.970504 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:28.988077 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 852/1741] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:28.995099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06fba9-458c-4a1e-b758-17fcad7f433c admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:29.165501 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06fba9-458c-4a1e-b758-17fcad7f433c admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:29.181967 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae124c-7753-4037-9dfb-feb56819b919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:29.182695 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae124c-7753-4037-9dfb-feb56819b919 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:29.182970 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae124c-7753-4037-9dfb-feb56819b919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:29.183246 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae124c-7753-4037-9dfb-feb56819b919 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:29.188683 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae124c-7753-4037-9dfb-feb56819b919 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:29.190776 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae124c-7753-4037-9dfb-feb56819b919 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:29.257983 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 890/1742] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:32:28 2022] POST /identity/v3/projects => generated 379 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:29.262865 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a72bde-9dec-46d2-ab9c-aea3b92cea4a None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:29.290039 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a72bde-9dec-46d2-ab9c-aea3b92cea4a None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:29.461692 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216dd94d-7ce9-4199-9412-be0fefb1fb18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:29.462861 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216dd94d-7ce9-4199-9412-be0fefb1fb18 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:29.463262 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216dd94d-7ce9-4199-9412-be0fefb1fb18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:29.463628 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216dd94d-7ce9-4199-9412-be0fefb1fb18 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:29.464492 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216dd94d-7ce9-4199-9412-be0fefb1fb18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:29.472771 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216dd94d-7ce9-4199-9412-be0fefb1fb18 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:29.534213 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 853/1743] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:32:29 2022] POST /identity/v3/users => generated 296 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:29.537579 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae124c-7753-4037-9dfb-feb56819b919 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:29.550270 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae124c-7753-4037-9dfb-feb56819b919 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:29.563676 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee17efc-b699-47cb-b0b0-0f9fac3a7c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:29.565795 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee17efc-b699-47cb-b0b0-0f9fac3a7c51 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:29.568418 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee17efc-b699-47cb-b0b0-0f9fac3a7c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:29.571610 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee17efc-b699-47cb-b0b0-0f9fac3a7c51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:29.572698 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee17efc-b699-47cb-b0b0-0f9fac3a7c51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:29.580899 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee17efc-b699-47cb-b0b0-0f9fac3a7c51 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:29.614900 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216dd94d-7ce9-4199-9412-be0fefb1fb18 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:29.624382 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 891/1744] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:29.631675 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216dd94d-7ce9-4199-9412-be0fefb1fb18 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:29.645492 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed68fb7-0270-493e-9c65-b60b77df5023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:29.646692 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed68fb7-0270-493e-9c65-b60b77df5023 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:29.646978 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed68fb7-0270-493e-9c65-b60b77df5023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:29.647269 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed68fb7-0270-493e-9c65-b60b77df5023 admin admin] PATH_INFO: `/v3/projects/4c6a6ba4305b465586657292930f3a3f/users/1e121be54b704364b14539a19b643da3/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:29.695989 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed68fb7-0270-493e-9c65-b60b77df5023 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c6a6ba4305b465586657292930f3a3f, user_id=1e121be54b704364b14539a19b643da3, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:29.700652 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed68fb7-0270-493e-9c65-b60b77df5023 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:29.754987 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee17efc-b699-47cb-b0b0-0f9fac3a7c51 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:29.755934 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 854/1745] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:29 2022] PUT /identity/v3/projects/4c6a6ba4305b465586657292930f3a3f/users/1e121be54b704364b14539a19b643da3/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:29.764628 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:29.793495 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee17efc-b699-47cb-b0b0-0f9fac3a7c51 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:29.977868 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c14e8de-c39d-4ed9-9a1b-f55712ad6e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:29.979208 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14e8de-c39d-4ed9-9a1b-f55712ad6e5a admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:29.979612 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14e8de-c39d-4ed9-9a1b-f55712ad6e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:29.980181 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14e8de-c39d-4ed9-9a1b-f55712ad6e5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:29.981727 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c14e8de-c39d-4ed9-9a1b-f55712ad6e5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:29.984293 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c14e8de-c39d-4ed9-9a1b-f55712ad6e5a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:30.005754 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed68fb7-0270-493e-9c65-b60b77df5023 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:30.017148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 892/1746] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:30.044476 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed68fb7-0270-493e-9c65-b60b77df5023 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:30.063735 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb8a78-bc3b-4028-b67d-10d209ba5dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:30.074032 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb8a78-bc3b-4028-b67d-10d209ba5dd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:30.074490 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb8a78-bc3b-4028-b67d-10d209ba5dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:30.074779 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb8a78-bc3b-4028-b67d-10d209ba5dd9 admin admin] PATH_INFO: `/v3/projects/4c6a6ba4305b465586657292930f3a3f/users/1e121be54b704364b14539a19b643da3/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:30.077526 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eb8a78-bc3b-4028-b67d-10d209ba5dd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c6a6ba4305b465586657292930f3a3f, user_id=1e121be54b704364b14539a19b643da3, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:30.082811 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eb8a78-bc3b-4028-b67d-10d209ba5dd9 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:30.117392 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaeda80-e172-454f-a4ac-a7b61b698b24 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:30.117783 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaeda80-e172-454f-a4ac-a7b61b698b24 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:30.118075 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaeda80-e172-454f-a4ac-a7b61b698b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:30.126373 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 855/1747] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:30 2022] PUT /identity/v3/projects/4c6a6ba4305b465586657292930f3a3f/users/1e121be54b704364b14539a19b643da3/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:32:30.150501 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-fbaeda80-e172-454f-a4ac-a7b61b698b24 None None] MFA Rules not processed for user `1e121be54b704364b14539a19b643da3`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:30.151739 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fbaeda80-e172-454f-a4ac-a7b61b698b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:30.363333 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 893/1748] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:30 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:30.379785 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb8a78-bc3b-4028-b67d-10d209ba5dd9 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:30.382467 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-03eb8a78-bc3b-4028-b67d-10d209ba5dd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:30.631553 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb8a78-bc3b-4028-b67d-10d209ba5dd9 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:30.646499 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2944a95-0e3b-45a5-be0b-29e01e5dab01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:30.647177 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2944a95-0e3b-45a5-be0b-29e01e5dab01 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:30.647466 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2944a95-0e3b-45a5-be0b-29e01e5dab01 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:30.647749 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2944a95-0e3b-45a5-be0b-29e01e5dab01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:30.696369 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2944a95-0e3b-45a5-be0b-29e01e5dab01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:30.710999 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2944a95-0e3b-45a5-be0b-29e01e5dab01 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:30.746902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 856/1749] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:30 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:31.097027 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaeda80-e172-454f-a4ac-a7b61b698b24 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:31.099812 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fbaeda80-e172-454f-a4ac-a7b61b698b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:31.283946 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaeda80-e172-454f-a4ac-a7b61b698b24 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:31.297618 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea26710-fc73-41b2-9921-5b2f5a9a80e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:31.298608 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea26710-fc73-41b2-9921-5b2f5a9a80e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:31.303142 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea26710-fc73-41b2-9921-5b2f5a9a80e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:31.305555 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea26710-fc73-41b2-9921-5b2f5a9a80e5 admin admin] PATH_INFO: `/v3/projects/6402175446b748e5a4750f360e80ffa4` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:31.307346 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea26710-fc73-41b2-9921-5b2f5a9a80e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6402175446b748e5a4750f360e80ffa4)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:31.309310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea26710-fc73-41b2-9921-5b2f5a9a80e5 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:31.428506 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:31.481612 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-aea26710-fc73-41b2-9921-5b2f5a9a80e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6402175446b748e5a4750f360e80ffa4', 'request_id': 'req-aea26710-fc73-41b2-9921-5b2f5a9a80e5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:31.527546 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-aea26710-fc73-41b2-9921-5b2f5a9a80e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6402175446b748e5a4750f360e80ffa4', 'request_id': 'req-aea26710-fc73-41b2-9921-5b2f5a9a80e5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:31.554207 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-aea26710-fc73-41b2-9921-5b2f5a9a80e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6402175446b748e5a4750f360e80ffa4', 'request_id': 'req-aea26710-fc73-41b2-9921-5b2f5a9a80e5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:31.594688 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-aea26710-fc73-41b2-9921-5b2f5a9a80e5 admin admin] The token cache is being invalidate because project 6402175446b748e5a4750f360e80ffa4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:32:31.594984 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-aea26710-fc73-41b2-9921-5b2f5a9a80e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:31.597703 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-aea26710-fc73-41b2-9921-5b2f5a9a80e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:31.604974 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 894/1750] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:32:31 2022] DELETE /identity/v3/projects/6402175446b748e5a4750f360e80ffa4 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:31.609652 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2944a95-0e3b-45a5-be0b-29e01e5dab01 service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:31.632106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2944a95-0e3b-45a5-be0b-29e01e5dab01 service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:31.850637 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:31.856930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8709cb0-48f1-4634-9630-4a6aa091a9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:31.870860 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8709cb0-48f1-4634-9630-4a6aa091a9eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:31.871380 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8709cb0-48f1-4634-9630-4a6aa091a9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:31.872833 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8709cb0-48f1-4634-9630-4a6aa091a9eb admin admin] PATH_INFO: `/v3/users/664f02bdb65a4030960f5762a82d9c1e` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:31.875098 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8709cb0-48f1-4634-9630-4a6aa091a9eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=664f02bdb65a4030960f5762a82d9c1e)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:31.880562 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8709cb0-48f1-4634-9630-4a6aa091a9eb admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:32.026057 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea26710-fc73-41b2-9921-5b2f5a9a80e5 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:32.027563 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-aea26710-fc73-41b2-9921-5b2f5a9a80e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:32.057100 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e8709cb0-48f1-4634-9630-4a6aa091a9eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '664f02bdb65a4030960f5762a82d9c1e'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:32.058758 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e8709cb0-48f1-4634-9630-4a6aa091a9eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '664f02bdb65a4030960f5762a82d9c1e'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:32.180100 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e8709cb0-48f1-4634-9630-4a6aa091a9eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '664f02bdb65a4030960f5762a82d9c1e'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:32.222907 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e8709cb0-48f1-4634-9630-4a6aa091a9eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '664f02bdb65a4030960f5762a82d9c1e'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:32.249309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e8709cb0-48f1-4634-9630-4a6aa091a9eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '664f02bdb65a4030960f5762a82d9c1e'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:32.272759 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 857/1751] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:32:31 2022] DELETE /identity/v3/users/664f02bdb65a4030960f5762a82d9c1e => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:32.431559 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea26710-fc73-41b2-9921-5b2f5a9a80e5 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:32.823198 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ed44ba-bb6a-4f96-affb-20c2412d0db6 tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e121be54b704364b14539a19b643da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c6a6ba4305b465586657292930f3a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:32.824070 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed44ba-bb6a-4f96-affb-20c2412d0db6 tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:32.824459 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed44ba-bb6a-4f96-affb-20c2412d0db6 tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:32.824817 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed44ba-bb6a-4f96-affb-20c2412d0db6 tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:32.827910 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ed44ba-bb6a-4f96-affb-20c2412d0db6 tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:32:32.828245 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ed44ba-bb6a-4f96-affb-20c2412d0db6 tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:32.830316 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ed44ba-bb6a-4f96-affb-20c2412d0db6 tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:32.918640 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8709cb0-48f1-4634-9630-4a6aa091a9eb admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:32.919630 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 895/1752] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:32:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 893 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:32:32.921412 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e8709cb0-48f1-4634-9630-4a6aa091a9eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:33.000947 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8709cb0-48f1-4634-9630-4a6aa091a9eb admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:33.033092 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55323a2d-9ab7-4c09-9aa3-24939f2f55fc tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e121be54b704364b14539a19b643da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c6a6ba4305b465586657292930f3a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:33.037739 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55323a2d-9ab7-4c09-9aa3-24939f2f55fc tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:33.039908 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55323a2d-9ab7-4c09-9aa3-24939f2f55fc tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:33.040539 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55323a2d-9ab7-4c09-9aa3-24939f2f55fc tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:33.042617 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55323a2d-9ab7-4c09-9aa3-24939f2f55fc tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:33.056419 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55323a2d-9ab7-4c09-9aa3-24939f2f55fc tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:33.178099 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 858/1753] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:32:32 2022] POST /identity/v3/projects => generated 353 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:33.178696 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ed44ba-bb6a-4f96-affb-20c2412d0db6 tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:33.198459 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ed44ba-bb6a-4f96-affb-20c2412d0db6 tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:33.366633 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:33.383060 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4306c55f-5f5a-460c-ad60-8eb91d53a515 tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e121be54b704364b14539a19b643da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c6a6ba4305b465586657292930f3a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:33.383900 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306c55f-5f5a-460c-ad60-8eb91d53a515 tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:33.384189 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306c55f-5f5a-460c-ad60-8eb91d53a515 tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:33.384472 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306c55f-5f5a-460c-ad60-8eb91d53a515 tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:33.385229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4306c55f-5f5a-460c-ad60-8eb91d53a515 tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:32:33.385530 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4306c55f-5f5a-460c-ad60-8eb91d53a515 tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:33.391605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4306c55f-5f5a-460c-ad60-8eb91d53a515 tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:33.442673 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 896/1754] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:32:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:32:33.570283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55323a2d-9ab7-4c09-9aa3-24939f2f55fc tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:33.595184 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55323a2d-9ab7-4c09-9aa3-24939f2f55fc tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:33.604886 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1e179-7bec-4b09-bb9c-1095a23c6bc8 tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e121be54b704364b14539a19b643da3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c6a6ba4305b465586657292930f3a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:33.606147 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1e179-7bec-4b09-bb9c-1095a23c6bc8 tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:33.606463 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1e179-7bec-4b09-bb9c-1095a23c6bc8 tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:33.607657 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1e179-7bec-4b09-bb9c-1095a23c6bc8 tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] PATH_INFO: `/v3/projects/6190da97c7bf413f85f7854513fb63a3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:33.613527 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1e179-7bec-4b09-bb9c-1095a23c6bc8 tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] RBAC: Authorizing `identity:delete_project(project_id=6190da97c7bf413f85f7854513fb63a3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:33.620833 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1e179-7bec-4b09-bb9c-1095a23c6bc8 tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:33.775807 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-abf1e179-7bec-4b09-bb9c-1095a23c6bc8 tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6190da97c7bf413f85f7854513fb63a3', 'request_id': 'req-abf1e179-7bec-4b09-bb9c-1095a23c6bc8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:33.791499 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-abf1e179-7bec-4b09-bb9c-1095a23c6bc8 tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6190da97c7bf413f85f7854513fb63a3', 'request_id': 'req-abf1e179-7bec-4b09-bb9c-1095a23c6bc8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:33.817585 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-abf1e179-7bec-4b09-bb9c-1095a23c6bc8 tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6190da97c7bf413f85f7854513fb63a3', 'request_id': 'req-abf1e179-7bec-4b09-bb9c-1095a23c6bc8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:33.836680 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-abf1e179-7bec-4b09-bb9c-1095a23c6bc8 tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] The token cache is being invalidate because project 6190da97c7bf413f85f7854513fb63a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:32:33.836973 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-abf1e179-7bec-4b09-bb9c-1095a23c6bc8 tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:33.840648 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-abf1e179-7bec-4b09-bb9c-1095a23c6bc8 tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:33.860996 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 859/1755] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:32:33 2022] DELETE /identity/v3/projects/6190da97c7bf413f85f7854513fb63a3 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:34.437722 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4306c55f-5f5a-460c-ad60-8eb91d53a515 tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:34.446000 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4306c55f-5f5a-460c-ad60-8eb91d53a515 tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:34.643207 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1e179-7bec-4b09-bb9c-1095a23c6bc8 tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:34.644756 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-abf1e179-7bec-4b09-bb9c-1095a23c6bc8 tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:34.705260 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4306c55f-5f5a-460c-ad60-8eb91d53a515 tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:34.738724 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:34.824107 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1e179-7bec-4b09-bb9c-1095a23c6bc8 tempest-VolumeQuotaClassesTest-310441144 tempest-VolumeQuotaClassesTest-310441144-project-admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:34.835898 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d7b92f-5132-40d2-933d-8b08c08f08f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:34.836965 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7b92f-5132-40d2-933d-8b08c08f08f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:34.837117 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7b92f-5132-40d2-933d-8b08c08f08f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:34.837398 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7b92f-5132-40d2-933d-8b08c08f08f5 admin admin] PATH_INFO: `/v3/users/d72dd5014aa94f03a9ebfaaeea053710` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:34.838967 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d7b92f-5132-40d2-933d-8b08c08f08f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d72dd5014aa94f03a9ebfaaeea053710)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:34.851950 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d7b92f-5132-40d2-933d-8b08c08f08f5 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:34.952333 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dbc53b-246b-4456-aee6-79a3b968c673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:34.954520 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dbc53b-246b-4456-aee6-79a3b968c673 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:34.954816 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dbc53b-246b-4456-aee6-79a3b968c673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:34.955705 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dbc53b-246b-4456-aee6-79a3b968c673 admin admin] PATH_INFO: `/v3/users/4377ac3e0932476e88bc9c91e5475c3d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:34.958618 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dbc53b-246b-4456-aee6-79a3b968c673 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4377ac3e0932476e88bc9c91e5475c3d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:34.966971 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dbc53b-246b-4456-aee6-79a3b968c673 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:34.973158 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b9d7b92f-5132-40d2-933d-8b08c08f08f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd72dd5014aa94f03a9ebfaaeea053710'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:34.974893 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b9d7b92f-5132-40d2-933d-8b08c08f08f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd72dd5014aa94f03a9ebfaaeea053710'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:35.024182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b9d7b92f-5132-40d2-933d-8b08c08f08f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd72dd5014aa94f03a9ebfaaeea053710'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:35.059084 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b9d7b92f-5132-40d2-933d-8b08c08f08f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd72dd5014aa94f03a9ebfaaeea053710'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:35.098590 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d9dbc53b-246b-4456-aee6-79a3b968c673 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4377ac3e0932476e88bc9c91e5475c3d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:35.099909 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b9d7b92f-5132-40d2-933d-8b08c08f08f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd72dd5014aa94f03a9ebfaaeea053710'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:35.146136 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 860/1756] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:32:34 2022] DELETE /identity/v3/users/d72dd5014aa94f03a9ebfaaeea053710 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:35.146623 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d9dbc53b-246b-4456-aee6-79a3b968c673 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4377ac3e0932476e88bc9c91e5475c3d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:35.147524 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d9dbc53b-246b-4456-aee6-79a3b968c673 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4377ac3e0932476e88bc9c91e5475c3d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:35.148585 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d9dbc53b-246b-4456-aee6-79a3b968c673 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4377ac3e0932476e88bc9c91e5475c3d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:35.152206 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c834d53-9edf-4926-8dce-a5508be0112d None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:35.152502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c834d53-9edf-4926-8dce-a5508be0112d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:35.152790 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c834d53-9edf-4926-8dce-a5508be0112d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:35.164773 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d9dbc53b-246b-4456-aee6-79a3b968c673 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4377ac3e0932476e88bc9c91e5475c3d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:35.193606 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 897/1757] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:32:34 2022] DELETE /identity/v3/users/4377ac3e0932476e88bc9c91e5475c3d => generated 0 bytes in 756 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:35.218304 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2742268-f676-4d96-8738-c3628b5a0687 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:35.218839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2742268-f676-4d96-8738-c3628b5a0687 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:35.218839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2742268-f676-4d96-8738-c3628b5a0687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:35.230021 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-5c834d53-9edf-4926-8dce-a5508be0112d None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:35.231513 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5c834d53-9edf-4926-8dce-a5508be0112d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:35.317853 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b2742268-f676-4d96-8738-c3628b5a0687 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:35.322871 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b2742268-f676-4d96-8738-c3628b5a0687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:35.330124 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 861/1758] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:35.331840 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651ce2f-3766-4676-8300-e6224d557a1c None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:35.332102 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651ce2f-3766-4676-8300-e6224d557a1c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:35.332384 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651ce2f-3766-4676-8300-e6224d557a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:35.364164 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 898/1759] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:35.369874 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba80e94-6dc4-417e-9e10-32d484d845ee None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:35.370758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba80e94-6dc4-417e-9e10-32d484d845ee None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:35.371194 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba80e94-6dc4-417e-9e10-32d484d845ee None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:35.381479 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 899/1760] 10.222.0.65 () {54 vars in 921 bytes} [Thu May 19 21:32:35 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:32:35.382579 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-3651ce2f-3766-4676-8300-e6224d557a1c None None] MFA Rules not processed for user `6d16feb169ac46f7983a6b8045de77c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:35.382696 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba80e94-6dc4-417e-9e10-32d484d845ee None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:35.383808 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3651ce2f-3766-4676-8300-e6224d557a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:35.395195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-bba80e94-6dc4-417e-9e10-32d484d845ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:35.457138 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba80e94-6dc4-417e-9e10-32d484d845ee None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:35.479127 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd75156-0e02-49d8-adc1-f0afed70bbf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:35.480890 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd75156-0e02-49d8-adc1-f0afed70bbf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:35.481172 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd75156-0e02-49d8-adc1-f0afed70bbf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:35.481462 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd75156-0e02-49d8-adc1-f0afed70bbf9 admin admin] PATH_INFO: `/v3/projects/80df3b105c9c44debf9d9842ac87e58d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:35.485403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd75156-0e02-49d8-adc1-f0afed70bbf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80df3b105c9c44debf9d9842ac87e58d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:35.487621 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd75156-0e02-49d8-adc1-f0afed70bbf9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:35.498637 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:35.523107 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:35.569808 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 862/1761] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:35.571261 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905b6f3-8051-4e52-8618-1899e657bc5e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:35.571548 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905b6f3-8051-4e52-8618-1899e657bc5e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:35.571832 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905b6f3-8051-4e52-8618-1899e657bc5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:35.619325 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-e905b6f3-8051-4e52-8618-1899e657bc5e None None] MFA Rules not processed for user `e254671b21fe4661a381b9521810cf80`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:35.624059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e905b6f3-8051-4e52-8618-1899e657bc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:35.658924 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cdd75156-0e02-49d8-adc1-f0afed70bbf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80df3b105c9c44debf9d9842ac87e58d', 'request_id': 'req-cdd75156-0e02-49d8-adc1-f0afed70bbf9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:35.684453 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cdd75156-0e02-49d8-adc1-f0afed70bbf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80df3b105c9c44debf9d9842ac87e58d', 'request_id': 'req-cdd75156-0e02-49d8-adc1-f0afed70bbf9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:35.712095 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cdd75156-0e02-49d8-adc1-f0afed70bbf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80df3b105c9c44debf9d9842ac87e58d', 'request_id': 'req-cdd75156-0e02-49d8-adc1-f0afed70bbf9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:35.779948 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cdd75156-0e02-49d8-adc1-f0afed70bbf9 admin admin] The token cache is being invalidate because project 80df3b105c9c44debf9d9842ac87e58d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:32:35.782453 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cdd75156-0e02-49d8-adc1-f0afed70bbf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:35.785874 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cdd75156-0e02-49d8-adc1-f0afed70bbf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:35.794473 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 900/1762] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:32:35 2022] DELETE /identity/v3/projects/80df3b105c9c44debf9d9842ac87e58d => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:35.796937 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd75156-0e02-49d8-adc1-f0afed70bbf9 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:35.799694 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cdd75156-0e02-49d8-adc1-f0afed70bbf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:35.855410 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 863/1763] 10.222.0.65 () {58 vars in 1018 bytes} [Thu May 19 21:32:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:35.855410 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e905b6f3-8051-4e52-8618-1899e657bc5e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:35.861594 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e905b6f3-8051-4e52-8618-1899e657bc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:36.044000 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e905b6f3-8051-4e52-8618-1899e657bc5e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:36.057594 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438db971-1967-4e75-85f6-edd499c4256c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:36.062561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438db971-1967-4e75-85f6-edd499c4256c service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:36.062960 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438db971-1967-4e75-85f6-edd499c4256c service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:36.063253 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438db971-1967-4e75-85f6-edd499c4256c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:36.067862 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-438db971-1967-4e75-85f6-edd499c4256c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:36.114619 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd75156-0e02-49d8-adc1-f0afed70bbf9 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:36.143054 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db01375-73b7-455c-9dab-25807e26388f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:36.143803 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db01375-73b7-455c-9dab-25807e26388f service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:36.144237 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db01375-73b7-455c-9dab-25807e26388f service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:36.145115 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db01375-73b7-455c-9dab-25807e26388f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:36.151503 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9db01375-73b7-455c-9dab-25807e26388f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:36.261900 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438db971-1967-4e75-85f6-edd499c4256c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:36.266694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438db971-1967-4e75-85f6-edd499c4256c service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:36.293900 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 864/1764] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:36.293900 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438db971-1967-4e75-85f6-edd499c4256c service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:36.315181 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db01375-73b7-455c-9dab-25807e26388f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:36.317228 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db01375-73b7-455c-9dab-25807e26388f service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:36.317843 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438db971-1967-4e75-85f6-edd499c4256c service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:36.328220 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f75f0-fefc-45b0-a06b-ac845aa97517 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:36.329679 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f75f0-fefc-45b0-a06b-ac845aa97517 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:36.331288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f75f0-fefc-45b0-a06b-ac845aa97517 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:36.331700 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f75f0-fefc-45b0-a06b-ac845aa97517 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:36.336303 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-860f75f0-fefc-45b0-a06b-ac845aa97517 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:36.341016 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 901/1765] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:36.342956 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b349b-864c-4092-9db2-145fbec2ece1 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:36.344057 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b349b-864c-4092-9db2-145fbec2ece1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:36.345736 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b349b-864c-4092-9db2-145fbec2ece1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:36.364371 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f75f0-fefc-45b0-a06b-ac845aa97517 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:36.367731 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f75f0-fefc-45b0-a06b-ac845aa97517 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:36.389383 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-8c7b349b-864c-4092-9db2-145fbec2ece1 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:36.392405 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8c7b349b-864c-4092-9db2-145fbec2ece1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:36.397730 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 865/1766] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:36.398683 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f75f0-fefc-45b0-a06b-ac845aa97517 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:36.401453 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-860f75f0-fefc-45b0-a06b-ac845aa97517 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:36.416356 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 902/1767] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:36.417425 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7b349b-864c-4092-9db2-145fbec2ece1 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:36.422072 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8c7b349b-864c-4092-9db2-145fbec2ece1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:36.428599 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f75f0-fefc-45b0-a06b-ac845aa97517 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:36.443891 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e11b8e9-4288-4083-b099-0fc07b4e2222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:36.444862 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11b8e9-4288-4083-b099-0fc07b4e2222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:36.447611 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11b8e9-4288-4083-b099-0fc07b4e2222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:36.447611 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11b8e9-4288-4083-b099-0fc07b4e2222 admin admin] PATH_INFO: `/v3/projects/8afcda833ca0433e95d6f9de6fc0df21` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:36.451044 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e11b8e9-4288-4083-b099-0fc07b4e2222 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8afcda833ca0433e95d6f9de6fc0df21)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:36.460078 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e11b8e9-4288-4083-b099-0fc07b4e2222 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:36.463555 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7b349b-864c-4092-9db2-145fbec2ece1 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:36.465649 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:36.475760 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64834087-fc3a-4f2d-9735-bec1a660b5ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:36.478526 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64834087-fc3a-4f2d-9735-bec1a660b5ff service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:36.478844 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64834087-fc3a-4f2d-9735-bec1a660b5ff service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:36.479764 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64834087-fc3a-4f2d-9735-bec1a660b5ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:36.517107 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64834087-fc3a-4f2d-9735-bec1a660b5ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:36.521415 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64834087-fc3a-4f2d-9735-bec1a660b5ff service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:36.562944 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 903/1768] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:36.574488 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75b32f-fa8c-4183-9290-39f043c218ac None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:36.574488 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75b32f-fa8c-4183-9290-39f043c218ac None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:36.574808 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75b32f-fa8c-4183-9290-39f043c218ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:36.637061 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2e11b8e9-4288-4083-b099-0fc07b4e2222 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8afcda833ca0433e95d6f9de6fc0df21', 'request_id': 'req-2e11b8e9-4288-4083-b099-0fc07b4e2222'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:36.654780 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-2a75b32f-fa8c-4183-9290-39f043c218ac None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:36.657588 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2a75b32f-fa8c-4183-9290-39f043c218ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:36.659216 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2e11b8e9-4288-4083-b099-0fc07b4e2222 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8afcda833ca0433e95d6f9de6fc0df21', 'request_id': 'req-2e11b8e9-4288-4083-b099-0fc07b4e2222'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:36.671416 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:36.674807 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2e11b8e9-4288-4083-b099-0fc07b4e2222 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8afcda833ca0433e95d6f9de6fc0df21', 'request_id': 'req-2e11b8e9-4288-4083-b099-0fc07b4e2222'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:36.704377 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2e11b8e9-4288-4083-b099-0fc07b4e2222 admin admin] The token cache is being invalidate because project 8afcda833ca0433e95d6f9de6fc0df21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:32:36.704655 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2e11b8e9-4288-4083-b099-0fc07b4e2222 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:36.705380 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2e11b8e9-4288-4083-b099-0fc07b4e2222 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:36.710301 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 866/1769] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:32:36 2022] DELETE /identity/v3/projects/8afcda833ca0433e95d6f9de6fc0df21 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:36.715463 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d571b-32c8-4b4c-914e-99591d9564d2 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:36.715763 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d571b-32c8-4b4c-914e-99591d9564d2 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:36.716026 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d571b-32c8-4b4c-914e-99591d9564d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:36.771987 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-f85d571b-32c8-4b4c-914e-99591d9564d2 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:36.773194 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f85d571b-32c8-4b4c-914e-99591d9564d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:36.876091 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 904/1770] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:36.878431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a75b32f-fa8c-4183-9290-39f043c218ac None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:36.902448 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a75b32f-fa8c-4183-9290-39f043c218ac None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:36.913549 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5741b1e8-27d2-43f7-b6f0-6aba9a996dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:36.914948 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741b1e8-27d2-43f7-b6f0-6aba9a996dbb admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:36.915408 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741b1e8-27d2-43f7-b6f0-6aba9a996dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:36.915878 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741b1e8-27d2-43f7-b6f0-6aba9a996dbb admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:36.917135 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5741b1e8-27d2-43f7-b6f0-6aba9a996dbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:32:36.917804 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5741b1e8-27d2-43f7-b6f0-6aba9a996dbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:36.919732 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5741b1e8-27d2-43f7-b6f0-6aba9a996dbb admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:36.944679 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 905/1771] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:32:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:32:36.949773 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5741b1e8-27d2-43f7-b6f0-6aba9a996dbb admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:36.972429 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5741b1e8-27d2-43f7-b6f0-6aba9a996dbb admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:36.981299 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd7432-d934-4b08-b298-fa4da7f78669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:36.983068 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd7432-d934-4b08-b298-fa4da7f78669 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:36.983523 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd7432-d934-4b08-b298-fa4da7f78669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:36.984047 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd7432-d934-4b08-b298-fa4da7f78669 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:36.985338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cd7432-d934-4b08-b298-fa4da7f78669 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:36.989830 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cd7432-d934-4b08-b298-fa4da7f78669 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:37.049219 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 906/1772] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:32:36 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:32:37.054174 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd7432-d934-4b08-b298-fa4da7f78669 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:37.060843 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 867/1773] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:32:37.072974 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533df6a-d639-4d0b-9069-d7c0fab353c2 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:37.073611 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533df6a-d639-4d0b-9069-d7c0fab353c2 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:37.074070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533df6a-d639-4d0b-9069-d7c0fab353c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:37.117615 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b533df6a-d639-4d0b-9069-d7c0fab353c2 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:37.119000 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b533df6a-d639-4d0b-9069-d7c0fab353c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:37.191697 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd7432-d934-4b08-b298-fa4da7f78669 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:37.199480 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdd7f75-7e1d-4b56-9975-82c3e5cf266f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:37.200372 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd7f75-7e1d-4b56-9975-82c3e5cf266f admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:37.200693 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd7f75-7e1d-4b56-9975-82c3e5cf266f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:37.210129 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd7f75-7e1d-4b56-9975-82c3e5cf266f admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:37.211414 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdd7f75-7e1d-4b56-9975-82c3e5cf266f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:37.214515 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdd7f75-7e1d-4b56-9975-82c3e5cf266f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:37.265229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 907/1774] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:32:37 2022] POST /identity/v3/users => generated 294 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:37.269658 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdd7f75-7e1d-4b56-9975-82c3e5cf266f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:37.326449 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdd7f75-7e1d-4b56-9975-82c3e5cf266f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:37.351473 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 868/1775] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:37.353696 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b533df6a-d639-4d0b-9069-d7c0fab353c2 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:37.358498 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b4557-a3b6-45a4-ab7f-2b6fa802cbb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:37.359755 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b4557-a3b6-45a4-ab7f-2b6fa802cbb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:37.363259 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b4557-a3b6-45a4-ab7f-2b6fa802cbb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:37.363787 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b4557-a3b6-45a4-ab7f-2b6fa802cbb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:37.365064 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b4557-a3b6-45a4-ab7f-2b6fa802cbb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:37.371986 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b4557-a3b6-45a4-ab7f-2b6fa802cbb2 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:37.390116 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b533df6a-d639-4d0b-9069-d7c0fab353c2 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:37.408827 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d64d076-3918-4765-89ea-2ca4da6511f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:37.410158 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64d076-3918-4765-89ea-2ca4da6511f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:37.417297 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 908/1776] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:37.419489 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b4557-a3b6-45a4-ab7f-2b6fa802cbb2 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:37.423483 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64d076-3918-4765-89ea-2ca4da6511f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:37.423828 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64d076-3918-4765-89ea-2ca4da6511f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:37.424657 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d64d076-3918-4765-89ea-2ca4da6511f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:32:37.424972 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d64d076-3918-4765-89ea-2ca4da6511f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:37.428402 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d64d076-3918-4765-89ea-2ca4da6511f3 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:37.448511 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 869/1777] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:32:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:32:37.449979 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b4557-a3b6-45a4-ab7f-2b6fa802cbb2 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:37.451087 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d64d076-3918-4765-89ea-2ca4da6511f3 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:37.467250 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b80feee-6156-4afb-b2d4-fb7801a19ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:37.468040 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80feee-6156-4afb-b2d4-fb7801a19ce1 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:37.468329 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80feee-6156-4afb-b2d4-fb7801a19ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:37.468610 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80feee-6156-4afb-b2d4-fb7801a19ce1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:37.469360 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b80feee-6156-4afb-b2d4-fb7801a19ce1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:37.470354 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d64d076-3918-4765-89ea-2ca4da6511f3 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:37.471349 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b80feee-6156-4afb-b2d4-fb7801a19ce1 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:37.482868 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5cbd79-bd95-4085-90a9-5df45ec9a802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:37.483885 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5cbd79-bd95-4085-90a9-5df45ec9a802 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:37.484167 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5cbd79-bd95-4085-90a9-5df45ec9a802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:37.484524 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5cbd79-bd95-4085-90a9-5df45ec9a802 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:37.485327 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5cbd79-bd95-4085-90a9-5df45ec9a802 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:37.487962 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5cbd79-bd95-4085-90a9-5df45ec9a802 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:37.502641 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 909/1778] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:37.507184 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b80feee-6156-4afb-b2d4-fb7801a19ce1 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:37.523802 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:37.524916 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b80feee-6156-4afb-b2d4-fb7801a19ce1 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:37.532589 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb9e26a-4fb3-4f9d-b6b2-a1795f5e37ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:37.535372 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9e26a-4fb3-4f9d-b6b2-a1795f5e37ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:37.536450 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9e26a-4fb3-4f9d-b6b2-a1795f5e37ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:37.537010 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9e26a-4fb3-4f9d-b6b2-a1795f5e37ca admin admin] PATH_INFO: `/v3/projects/700bbb441eec410ca0f39ba8854cbbb2/users/5b311ac6ba24492897bbd23593885f01/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:37.556440 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 870/1779] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:32:37 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:32:37.559127 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5cbd79-bd95-4085-90a9-5df45ec9a802 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:37.570178 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb9e26a-4fb3-4f9d-b6b2-a1795f5e37ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=700bbb441eec410ca0f39ba8854cbbb2, user_id=5b311ac6ba24492897bbd23593885f01, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:37.574025 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb9e26a-4fb3-4f9d-b6b2-a1795f5e37ca admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:37.600681 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 910/1780] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:37 2022] PUT /identity/v3/projects/700bbb441eec410ca0f39ba8854cbbb2/users/5b311ac6ba24492897bbd23593885f01/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:32:37.611567 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff6466-eb7e-42e2-b08e-2537191f7039 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:37.612251 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff6466-eb7e-42e2-b08e-2537191f7039 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:37.612788 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff6466-eb7e-42e2-b08e-2537191f7039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:37.670762 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-1dff6466-eb7e-42e2-b08e-2537191f7039 None None] MFA Rules not processed for user `5b311ac6ba24492897bbd23593885f01`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:37.672135 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1dff6466-eb7e-42e2-b08e-2537191f7039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:37.718199 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5cbd79-bd95-4085-90a9-5df45ec9a802 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:37.762442 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:37.984891 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a16652-ce67-4245-8e31-c1ab6b1303a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:37.985619 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a16652-ce67-4245-8e31-c1ab6b1303a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:37.988159 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a16652-ce67-4245-8e31-c1ab6b1303a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:37.990655 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a16652-ce67-4245-8e31-c1ab6b1303a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:37.992100 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a16652-ce67-4245-8e31-c1ab6b1303a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:37.999878 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a16652-ce67-4245-8e31-c1ab6b1303a1 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:38.023313 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 911/1781] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 412 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:32:38.026571 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e9a4e-fc6d-4011-9468-40ad224c89f7 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:38.026868 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e9a4e-fc6d-4011-9468-40ad224c89f7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:38.027146 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e9a4e-fc6d-4011-9468-40ad224c89f7 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:38.034211 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 912/1782] 10.222.0.65 () {54 vars in 921 bytes} [Thu May 19 21:32:38 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:32:38.037490 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e9a4e-fc6d-4011-9468-40ad224c89f7 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:38.081118 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e9a4e-fc6d-4011-9468-40ad224c89f7 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:38.088180 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 871/1783] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:32:37 2022] POST /identity/v3/users => generated 287 bytes in 529 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:38.088769 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a16652-ce67-4245-8e31-c1ab6b1303a1 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:38.101113 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb993b-f314-44c2-99f5-efe0626c1877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:38.101940 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb993b-f314-44c2-99f5-efe0626c1877 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:38.102221 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb993b-f314-44c2-99f5-efe0626c1877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:38.102520 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb993b-f314-44c2-99f5-efe0626c1877 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:38.107124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfb993b-f314-44c2-99f5-efe0626c1877 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:38.110375 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfb993b-f314-44c2-99f5-efe0626c1877 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:38.125856 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a16652-ce67-4245-8e31-c1ab6b1303a1 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:38.147259 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1182af7c-4851-4a77-bbee-0a3f39868598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:38.148612 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182af7c-4851-4a77-bbee-0a3f39868598 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:38.148909 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182af7c-4851-4a77-bbee-0a3f39868598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:38.149240 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182af7c-4851-4a77-bbee-0a3f39868598 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:38.151117 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1182af7c-4851-4a77-bbee-0a3f39868598 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:38.152869 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1182af7c-4851-4a77-bbee-0a3f39868598 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:38.194222 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 872/1784] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:38.196272 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 913/1785] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:32:38 2022] POST /identity/v3/projects => generated 387 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:38.196887 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb78762-66e3-43dc-928a-149b696356c0 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:38.197189 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb78762-66e3-43dc-928a-149b696356c0 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:38.197298 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb993b-f314-44c2-99f5-efe0626c1877 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:38.197466 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb78762-66e3-43dc-928a-149b696356c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:38.253664 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-adb78762-66e3-43dc-928a-149b696356c0 None None] MFA Rules not processed for user `e254671b21fe4661a381b9521810cf80`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:38.255486 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-adb78762-66e3-43dc-928a-149b696356c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:38.384961 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb993b-f314-44c2-99f5-efe0626c1877 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:38.397831 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95591d1d-1380-425e-815b-fc49d69b58cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:38.401214 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95591d1d-1380-425e-815b-fc49d69b58cc admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:38.402053 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95591d1d-1380-425e-815b-fc49d69b58cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:38.403464 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95591d1d-1380-425e-815b-fc49d69b58cc admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:38.404556 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95591d1d-1380-425e-815b-fc49d69b58cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:38.411478 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95591d1d-1380-425e-815b-fc49d69b58cc admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:38.453864 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 873/1786] 10.222.0.65 () {58 vars in 1018 bytes} [Thu May 19 21:32:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:38.454793 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb78762-66e3-43dc-928a-149b696356c0 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:38.466728 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 914/1787] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:32:38 2022] POST /identity/v3/users => generated 300 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:38.468216 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb78762-66e3-43dc-928a-149b696356c0 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:38.471816 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95591d1d-1380-425e-815b-fc49d69b58cc admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:38.480411 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c9b82-728f-4ea6-a811-128ddbc095bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:38.484814 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c9b82-728f-4ea6-a811-128ddbc095bf admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:38.485318 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c9b82-728f-4ea6-a811-128ddbc095bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:38.485773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c9b82-728f-4ea6-a811-128ddbc095bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:38.486748 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361c9b82-728f-4ea6-a811-128ddbc095bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:38.491068 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361c9b82-728f-4ea6-a811-128ddbc095bf admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:38.501946 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95591d1d-1380-425e-815b-fc49d69b58cc admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:38.514295 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb878c4-63fc-4adc-9f9b-cc8bf83d23ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:38.515472 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb878c4-63fc-4adc-9f9b-cc8bf83d23ff admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:38.517078 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb878c4-63fc-4adc-9f9b-cc8bf83d23ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:38.518032 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb878c4-63fc-4adc-9f9b-cc8bf83d23ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:38.521777 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb878c4-63fc-4adc-9f9b-cc8bf83d23ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:38.526426 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 874/1788] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:38.527840 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb878c4-63fc-4adc-9f9b-cc8bf83d23ff admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:38.530515 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c9b82-728f-4ea6-a811-128ddbc095bf admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:38.546979 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:38.556479 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c9b82-728f-4ea6-a811-128ddbc095bf admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:38.563756 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 915/1789] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:38.568096 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb878c4-63fc-4adc-9f9b-cc8bf83d23ff admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:38.580255 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa27602-fc47-4cc7-b99f-98e13e52853d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:38.583526 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa27602-fc47-4cc7-b99f-98e13e52853d admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:38.585591 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa27602-fc47-4cc7-b99f-98e13e52853d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:38.586446 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa27602-fc47-4cc7-b99f-98e13e52853d admin admin] PATH_INFO: `/v3/projects/799367c091bf4817ae4af6651f3cd47f/users/d87dc63cdfc44556b92138b85a4dd49f/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:38.595346 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb878c4-63fc-4adc-9f9b-cc8bf83d23ff admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:38.612512 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2697165e-2eb5-4614-b4fd-d91a6d9ef442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:38.613490 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697165e-2eb5-4614-b4fd-d91a6d9ef442 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:38.615198 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697165e-2eb5-4614-b4fd-d91a6d9ef442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:38.615505 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697165e-2eb5-4614-b4fd-d91a6d9ef442 admin admin] PATH_INFO: `/v3/projects/ce0653e5f17c46aca953f344126c0e91/users/34255f546ba34801aeeb33e2677b1e6a/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:38.624953 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa27602-fc47-4cc7-b99f-98e13e52853d admin admin] RBAC: Authorizing `identity:create_grant(project_id=799367c091bf4817ae4af6651f3cd47f, user_id=d87dc63cdfc44556b92138b85a4dd49f, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:38.638320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa27602-fc47-4cc7-b99f-98e13e52853d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:38.659196 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2697165e-2eb5-4614-b4fd-d91a6d9ef442 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce0653e5f17c46aca953f344126c0e91, user_id=34255f546ba34801aeeb33e2677b1e6a, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:38.674427 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2697165e-2eb5-4614-b4fd-d91a6d9ef442 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:38.675123 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 875/1790] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:38 2022] PUT /identity/v3/projects/799367c091bf4817ae4af6651f3cd47f/users/d87dc63cdfc44556b92138b85a4dd49f/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:38.691554 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f6372-9e58-45d4-aea8-86758588b9ce None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:38.694181 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f6372-9e58-45d4-aea8-86758588b9ce None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:38.694901 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f6372-9e58-45d4-aea8-86758588b9ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:38.705305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 916/1791] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:38 2022] PUT /identity/v3/projects/ce0653e5f17c46aca953f344126c0e91/users/34255f546ba34801aeeb33e2677b1e6a/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:38.713018 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2697165e-2eb5-4614-b4fd-d91a6d9ef442 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:38.739326 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-2a2f6372-9e58-45d4-aea8-86758588b9ce None None] MFA Rules not processed for user `d87dc63cdfc44556b92138b85a4dd49f`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:38.741779 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-2a2f6372-9e58-45d4-aea8-86758588b9ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:38.811670 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:38.843348 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2697165e-2eb5-4614-b4fd-d91a6d9ef442 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:38.856664 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f428d-4fd9-4e3d-884b-12efc7938c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:38.857741 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f428d-4fd9-4e3d-884b-12efc7938c9d admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:38.857930 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f428d-4fd9-4e3d-884b-12efc7938c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:38.858204 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f428d-4fd9-4e3d-884b-12efc7938c9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:38.858985 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845f428d-4fd9-4e3d-884b-12efc7938c9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:38.861501 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845f428d-4fd9-4e3d-884b-12efc7938c9d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:38.881825 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 917/1792] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:38.897429 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f428d-4fd9-4e3d-884b-12efc7938c9d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:38.897528 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 876/1793] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:38.918418 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f6372-9e58-45d4-aea8-86758588b9ce None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:38.920106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-2a2f6372-9e58-45d4-aea8-86758588b9ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:38.920548 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f428d-4fd9-4e3d-884b-12efc7938c9d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:38.927289 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324774e-a842-40ac-99b6-a86438a43fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:38.928229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324774e-a842-40ac-99b6-a86438a43fa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:38.928510 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324774e-a842-40ac-99b6-a86438a43fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:38.928835 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324774e-a842-40ac-99b6-a86438a43fa7 admin admin] PATH_INFO: `/v3/projects/ce0653e5f17c46aca953f344126c0e91/users/34255f546ba34801aeeb33e2677b1e6a/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:38.931647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e324774e-a842-40ac-99b6-a86438a43fa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce0653e5f17c46aca953f344126c0e91, user_id=34255f546ba34801aeeb33e2677b1e6a, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:38.934551 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e324774e-a842-40ac-99b6-a86438a43fa7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:38.966370 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 918/1794] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:38 2022] PUT /identity/v3/projects/ce0653e5f17c46aca953f344126c0e91/users/34255f546ba34801aeeb33e2677b1e6a/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:38.975072 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21794c0d-55b8-4d05-95ad-a568e98f9e8e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:38.975072 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21794c0d-55b8-4d05-95ad-a568e98f9e8e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:38.975372 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21794c0d-55b8-4d05-95ad-a568e98f9e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:39.010375 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-21794c0d-55b8-4d05-95ad-a568e98f9e8e None None] MFA Rules not processed for user `34255f546ba34801aeeb33e2677b1e6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:39.011749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-21794c0d-55b8-4d05-95ad-a568e98f9e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:39.048594 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f6372-9e58-45d4-aea8-86758588b9ce None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:39.110578 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 919/1795] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:38 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:39.123355 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21794c0d-55b8-4d05-95ad-a568e98f9e8e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:39.125296 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-21794c0d-55b8-4d05-95ad-a568e98f9e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:39.142856 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e582ccf1-0bed-4381-858c-d76fd3306d77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:39.146266 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e582ccf1-0bed-4381-858c-d76fd3306d77 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:39.146832 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e582ccf1-0bed-4381-858c-d76fd3306d77 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:39.147447 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e582ccf1-0bed-4381-858c-d76fd3306d77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:39.159809 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:39.173075 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e582ccf1-0bed-4381-858c-d76fd3306d77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:39.175884 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e582ccf1-0bed-4381-858c-d76fd3306d77 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:39.204473 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21794c0d-55b8-4d05-95ad-a568e98f9e8e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:39.207580 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 877/1796] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:39.208511 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e09fc1-e483-4c7f-a932-4fb393845756 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:39.209197 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e09fc1-e483-4c7f-a932-4fb393845756 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:39.209478 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e09fc1-e483-4c7f-a932-4fb393845756 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:39.209753 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e09fc1-e483-4c7f-a932-4fb393845756 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:39.231918 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e09fc1-e483-4c7f-a932-4fb393845756 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:39.233521 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e09fc1-e483-4c7f-a932-4fb393845756 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:39.253460 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e582ccf1-0bed-4381-858c-d76fd3306d77 service nova] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:39.259255 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 920/1797] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:39 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:39.267915 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e582ccf1-0bed-4381-858c-d76fd3306d77 service nova] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:39.279141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea038057-0fd9-455c-a9cc-0012b3751701 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:39.280828 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea038057-0fd9-455c-a9cc-0012b3751701 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:39.281846 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea038057-0fd9-455c-a9cc-0012b3751701 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:39.291049 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea038057-0fd9-455c-a9cc-0012b3751701 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:39.340199 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e09fc1-e483-4c7f-a932-4fb393845756 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:39.389775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea038057-0fd9-455c-a9cc-0012b3751701 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:39.392161 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea038057-0fd9-455c-a9cc-0012b3751701 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:39.424592 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 878/1798] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:39.427953 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:39.459056 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e09fc1-e483-4c7f-a932-4fb393845756 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:39.465368 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1f08a7-1268-4bce-9d24-76a1d2b100ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:39.466697 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1f08a7-1268-4bce-9d24-76a1d2b100ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:39.467092 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1f08a7-1268-4bce-9d24-76a1d2b100ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:39.467438 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1f08a7-1268-4bce-9d24-76a1d2b100ff admin admin] PATH_INFO: `/v3/users/5b311ac6ba24492897bbd23593885f01` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:39.469579 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1f08a7-1268-4bce-9d24-76a1d2b100ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b311ac6ba24492897bbd23593885f01)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:39.472255 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1f08a7-1268-4bce-9d24-76a1d2b100ff admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:39.514701 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea038057-0fd9-455c-a9cc-0012b3751701 service nova] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:39.528190 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ea038057-0fd9-455c-a9cc-0012b3751701 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:39.552812 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea038057-0fd9-455c-a9cc-0012b3751701 service nova] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:39.559821 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2601c652-dc4c-4313-85c0-4bbd6e1a6334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:39.560894 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2601c652-dc4c-4313-85c0-4bbd6e1a6334 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:39.561141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2601c652-dc4c-4313-85c0-4bbd6e1a6334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:39.561557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2601c652-dc4c-4313-85c0-4bbd6e1a6334 admin admin] PATH_INFO: `/v3/projects/0ceaaceb6bce449aa1ccd51de3a605ee` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:39.568714 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2601c652-dc4c-4313-85c0-4bbd6e1a6334 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ceaaceb6bce449aa1ccd51de3a605ee)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:39.571064 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2601c652-dc4c-4313-85c0-4bbd6e1a6334 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:39.630998 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3f1f08a7-1268-4bce-9d24-76a1d2b100ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b311ac6ba24492897bbd23593885f01'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:39.672352 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3f1f08a7-1268-4bce-9d24-76a1d2b100ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b311ac6ba24492897bbd23593885f01'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:39.673472 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3f1f08a7-1268-4bce-9d24-76a1d2b100ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b311ac6ba24492897bbd23593885f01'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:39.674274 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3f1f08a7-1268-4bce-9d24-76a1d2b100ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b311ac6ba24492897bbd23593885f01'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:39.686492 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3f1f08a7-1268-4bce-9d24-76a1d2b100ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b311ac6ba24492897bbd23593885f01'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:39.713582 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 921/1799] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:32:39 2022] DELETE /identity/v3/users/5b311ac6ba24492897bbd23593885f01 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:32:39.715539 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1f08a7-1268-4bce-9d24-76a1d2b100ff admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:39.718054 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3f1f08a7-1268-4bce-9d24-76a1d2b100ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:39.725732 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2601c652-dc4c-4313-85c0-4bbd6e1a6334 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ceaaceb6bce449aa1ccd51de3a605ee', 'request_id': 'req-2601c652-dc4c-4313-85c0-4bbd6e1a6334'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:39.761261 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2601c652-dc4c-4313-85c0-4bbd6e1a6334 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ceaaceb6bce449aa1ccd51de3a605ee', 'request_id': 'req-2601c652-dc4c-4313-85c0-4bbd6e1a6334'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:39.780019 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2601c652-dc4c-4313-85c0-4bbd6e1a6334 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ceaaceb6bce449aa1ccd51de3a605ee', 'request_id': 'req-2601c652-dc4c-4313-85c0-4bbd6e1a6334'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:39.803827 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2601c652-dc4c-4313-85c0-4bbd6e1a6334 admin admin] The token cache is being invalidate because project 0ceaaceb6bce449aa1ccd51de3a605ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:32:39.804283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2601c652-dc4c-4313-85c0-4bbd6e1a6334 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:39.805835 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2601c652-dc4c-4313-85c0-4bbd6e1a6334 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:39.813496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 879/1800] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:32:39 2022] DELETE /identity/v3/projects/0ceaaceb6bce449aa1ccd51de3a605ee => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:32:39.819940 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe60a73-9259-460a-923e-9a87b844fa25 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:39.821695 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe60a73-9259-460a-923e-9a87b844fa25 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:39.822084 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe60a73-9259-460a-923e-9a87b844fa25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:39.848914 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1f08a7-1268-4bce-9d24-76a1d2b100ff admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:39.853860 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a3893-5fe1-410c-bd57-eaad2471d8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:39.854808 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a3893-5fe1-410c-bd57-eaad2471d8b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:39.855086 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a3893-5fe1-410c-bd57-eaad2471d8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:39.855835 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a3893-5fe1-410c-bd57-eaad2471d8b9 admin admin] PATH_INFO: `/v3/projects/e0082fc7723648ccb2ba89573769debf` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:39.856743 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a3893-5fe1-410c-bd57-eaad2471d8b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0082fc7723648ccb2ba89573769debf)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:39.859309 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a3893-5fe1-410c-bd57-eaad2471d8b9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:39.923752 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-910a3893-5fe1-410c-bd57-eaad2471d8b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0082fc7723648ccb2ba89573769debf', 'request_id': 'req-910a3893-5fe1-410c-bd57-eaad2471d8b9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:39.932971 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-910a3893-5fe1-410c-bd57-eaad2471d8b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0082fc7723648ccb2ba89573769debf', 'request_id': 'req-910a3893-5fe1-410c-bd57-eaad2471d8b9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:39.945269 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-4fe60a73-9259-460a-923e-9a87b844fa25 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:39.947879 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4fe60a73-9259-460a-923e-9a87b844fa25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:39.964423 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-910a3893-5fe1-410c-bd57-eaad2471d8b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0082fc7723648ccb2ba89573769debf', 'request_id': 'req-910a3893-5fe1-410c-bd57-eaad2471d8b9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:40.010128 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-910a3893-5fe1-410c-bd57-eaad2471d8b9 admin admin] The token cache is being invalidate because project e0082fc7723648ccb2ba89573769debf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:32:40.010461 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-910a3893-5fe1-410c-bd57-eaad2471d8b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:40.013300 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-910a3893-5fe1-410c-bd57-eaad2471d8b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:40.023176 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 922/1801] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:32:39 2022] DELETE /identity/v3/projects/e0082fc7723648ccb2ba89573769debf => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:40.024220 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a3893-5fe1-410c-bd57-eaad2471d8b9 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:40.025731 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-910a3893-5fe1-410c-bd57-eaad2471d8b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:40.032236 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:40.108254 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 880/1802] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:40.109373 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe60a73-9259-460a-923e-9a87b844fa25 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:40.116952 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4fe60a73-9259-460a-923e-9a87b844fa25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:40.125197 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a3893-5fe1-410c-bd57-eaad2471d8b9 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:40.134729 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3222ca8-ceb1-418c-a63a-f9febb36e54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:40.135642 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3222ca8-ceb1-418c-a63a-f9febb36e54f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:40.136107 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3222ca8-ceb1-418c-a63a-f9febb36e54f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:40.136577 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3222ca8-ceb1-418c-a63a-f9febb36e54f admin admin] PATH_INFO: `/v3/users/681894709e614c918144b6b405c23ee5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:40.138140 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3222ca8-ceb1-418c-a63a-f9febb36e54f admin admin] RBAC: Authorizing `identity:delete_user(user_id=681894709e614c918144b6b405c23ee5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:40.142850 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3222ca8-ceb1-418c-a63a-f9febb36e54f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:40.145837 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe60a73-9259-460a-923e-9a87b844fa25 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:40.159662 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d090acd-3cc6-4d88-9c81-94ba22e6596d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:40.160929 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d090acd-3cc6-4d88-9c81-94ba22e6596d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:40.161215 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d090acd-3cc6-4d88-9c81-94ba22e6596d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:40.161493 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d090acd-3cc6-4d88-9c81-94ba22e6596d admin admin] PATH_INFO: `/v3/users/1e121be54b704364b14539a19b643da3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:40.166065 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d090acd-3cc6-4d88-9c81-94ba22e6596d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e121be54b704364b14539a19b643da3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:40.169896 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d090acd-3cc6-4d88-9c81-94ba22e6596d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:40.191520 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:40.247404 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d3222ca8-ceb1-418c-a63a-f9febb36e54f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '681894709e614c918144b6b405c23ee5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:40.261405 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4d090acd-3cc6-4d88-9c81-94ba22e6596d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e121be54b704364b14539a19b643da3'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:40.263649 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4d090acd-3cc6-4d88-9c81-94ba22e6596d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e121be54b704364b14539a19b643da3'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:40.265935 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d3222ca8-ceb1-418c-a63a-f9febb36e54f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '681894709e614c918144b6b405c23ee5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:40.267310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d3222ca8-ceb1-418c-a63a-f9febb36e54f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '681894709e614c918144b6b405c23ee5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:40.268570 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d3222ca8-ceb1-418c-a63a-f9febb36e54f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '681894709e614c918144b6b405c23ee5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:40.284895 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d3222ca8-ceb1-418c-a63a-f9febb36e54f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '681894709e614c918144b6b405c23ee5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:40.287918 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4d090acd-3cc6-4d88-9c81-94ba22e6596d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e121be54b704364b14539a19b643da3'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:40.299693 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4d090acd-3cc6-4d88-9c81-94ba22e6596d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e121be54b704364b14539a19b643da3'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:40.311635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4d090acd-3cc6-4d88-9c81-94ba22e6596d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e121be54b704364b14539a19b643da3'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:40.315533 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 923/1803] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:32:40 2022] DELETE /identity/v3/users/681894709e614c918144b6b405c23ee5 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:40.315767 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3222ca8-ceb1-418c-a63a-f9febb36e54f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:40.321014 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d3222ca8-ceb1-418c-a63a-f9febb36e54f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:40.326277 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 881/1804] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:32:40 2022] DELETE /identity/v3/users/1e121be54b704364b14539a19b643da3 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:40.476443 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3222ca8-ceb1-418c-a63a-f9febb36e54f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:40.537561 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:40.558231 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58496386-4cbd-479f-bfb0-4e079401e0df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:40.560590 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58496386-4cbd-479f-bfb0-4e079401e0df service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:40.562630 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58496386-4cbd-479f-bfb0-4e079401e0df service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:40.563168 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58496386-4cbd-479f-bfb0-4e079401e0df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:40.570853 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-58496386-4cbd-479f-bfb0-4e079401e0df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:40.672805 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c315db-1fa5-4e7c-87a8-bda3111b88e2 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:40.673269 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c315db-1fa5-4e7c-87a8-bda3111b88e2 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:40.673748 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c315db-1fa5-4e7c-87a8-bda3111b88e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:40.701512 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-57c315db-1fa5-4e7c-87a8-bda3111b88e2 None None] MFA Rules not processed for user `f47e7f746a014cb0a35390e7e0ce435d`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:40.703522 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-57c315db-1fa5-4e7c-87a8-bda3111b88e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:40.752634 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58496386-4cbd-479f-bfb0-4e079401e0df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:40.757552 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58496386-4cbd-479f-bfb0-4e079401e0df service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:40.771715 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 924/1805] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:40.774777 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 882/1806] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:32:41.407106 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58496386-4cbd-479f-bfb0-4e079401e0df service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:41.408694 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-58496386-4cbd-479f-bfb0-4e079401e0df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:41.430052 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58496386-4cbd-479f-bfb0-4e079401e0df service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:41.451213 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15330faa-4948-4e9e-acd0-666b591ad4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:41.452952 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15330faa-4948-4e9e-acd0-666b591ad4fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:41.456020 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15330faa-4948-4e9e-acd0-666b591ad4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:41.462137 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15330faa-4948-4e9e-acd0-666b591ad4fa admin admin] PATH_INFO: `/v3/projects/4c6a6ba4305b465586657292930f3a3f` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:41.468385 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15330faa-4948-4e9e-acd0-666b591ad4fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c6a6ba4305b465586657292930f3a3f)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:41.471848 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15330faa-4948-4e9e-acd0-666b591ad4fa admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:41.577064 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-15330faa-4948-4e9e-acd0-666b591ad4fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c6a6ba4305b465586657292930f3a3f', 'request_id': 'req-15330faa-4948-4e9e-acd0-666b591ad4fa'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:41.588171 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-15330faa-4948-4e9e-acd0-666b591ad4fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c6a6ba4305b465586657292930f3a3f', 'request_id': 'req-15330faa-4948-4e9e-acd0-666b591ad4fa'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:41.604352 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-15330faa-4948-4e9e-acd0-666b591ad4fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c6a6ba4305b465586657292930f3a3f', 'request_id': 'req-15330faa-4948-4e9e-acd0-666b591ad4fa'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:41.635335 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-15330faa-4948-4e9e-acd0-666b591ad4fa admin admin] The token cache is being invalidate because project 4c6a6ba4305b465586657292930f3a3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:32:41.636330 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-15330faa-4948-4e9e-acd0-666b591ad4fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:41.637656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-15330faa-4948-4e9e-acd0-666b591ad4fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:41.641079 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 925/1807] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:32:41 2022] DELETE /identity/v3/projects/4c6a6ba4305b465586657292930f3a3f => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:41.645878 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69718e83-fbae-450f-a482-da23d80ebcd0 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:41.646433 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69718e83-fbae-450f-a482-da23d80ebcd0 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:41.646519 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69718e83-fbae-450f-a482-da23d80ebcd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:41.682493 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-69718e83-fbae-450f-a482-da23d80ebcd0 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:41.684311 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-69718e83-fbae-450f-a482-da23d80ebcd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:41.692382 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15330faa-4948-4e9e-acd0-666b591ad4fa admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:41.693977 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-15330faa-4948-4e9e-acd0-666b591ad4fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:41.841009 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15330faa-4948-4e9e-acd0-666b591ad4fa admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:41.861013 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 883/1808] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:32:41.866491 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69718e83-fbae-450f-a482-da23d80ebcd0 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:41.867224 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d98beda-53ea-4315-9803-d6f51682fb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:41.867224 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98beda-53ea-4315-9803-d6f51682fb01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:41.867224 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98beda-53ea-4315-9803-d6f51682fb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:41.867224 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98beda-53ea-4315-9803-d6f51682fb01 admin admin] PATH_INFO: `/v3/projects/fddb2c9bfa0e4d9c9cd2bdf2b28b1616` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:41.870613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-69718e83-fbae-450f-a482-da23d80ebcd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:41.879685 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d98beda-53ea-4315-9803-d6f51682fb01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fddb2c9bfa0e4d9c9cd2bdf2b28b1616)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:41.881655 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d98beda-53ea-4315-9803-d6f51682fb01 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:41.892256 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69718e83-fbae-450f-a482-da23d80ebcd0 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:41.905099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:41.920219 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b805090f-e2eb-4cd6-b0b8-6afafd5af146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:41.921085 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805090f-e2eb-4cd6-b0b8-6afafd5af146 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:41.921385 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805090f-e2eb-4cd6-b0b8-6afafd5af146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:41.921657 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805090f-e2eb-4cd6-b0b8-6afafd5af146 admin admin] PATH_INFO: `/v3/projects/700bbb441eec410ca0f39ba8854cbbb2` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:41.931297 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b805090f-e2eb-4cd6-b0b8-6afafd5af146 admin admin] RBAC: Authorizing `identity:delete_project(project_id=700bbb441eec410ca0f39ba8854cbbb2)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:41.933177 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b805090f-e2eb-4cd6-b0b8-6afafd5af146 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:41.984799 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:41.998358 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9d98beda-53ea-4315-9803-d6f51682fb01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fddb2c9bfa0e4d9c9cd2bdf2b28b1616', 'request_id': 'req-9d98beda-53ea-4315-9803-d6f51682fb01'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:42.011543 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b805090f-e2eb-4cd6-b0b8-6afafd5af146 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '700bbb441eec410ca0f39ba8854cbbb2', 'request_id': 'req-b805090f-e2eb-4cd6-b0b8-6afafd5af146'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:42.030114 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b805090f-e2eb-4cd6-b0b8-6afafd5af146 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '700bbb441eec410ca0f39ba8854cbbb2', 'request_id': 'req-b805090f-e2eb-4cd6-b0b8-6afafd5af146'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:42.043920 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b805090f-e2eb-4cd6-b0b8-6afafd5af146 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '700bbb441eec410ca0f39ba8854cbbb2', 'request_id': 'req-b805090f-e2eb-4cd6-b0b8-6afafd5af146'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:42.056603 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9d98beda-53ea-4315-9803-d6f51682fb01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fddb2c9bfa0e4d9c9cd2bdf2b28b1616', 'request_id': 'req-9d98beda-53ea-4315-9803-d6f51682fb01'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:42.066501 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b805090f-e2eb-4cd6-b0b8-6afafd5af146 admin admin] The token cache is being invalidate because project 700bbb441eec410ca0f39ba8854cbbb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:32:42.067445 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b805090f-e2eb-4cd6-b0b8-6afafd5af146 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:42.068251 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9d98beda-53ea-4315-9803-d6f51682fb01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fddb2c9bfa0e4d9c9cd2bdf2b28b1616', 'request_id': 'req-9d98beda-53ea-4315-9803-d6f51682fb01'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:42.068947 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b805090f-e2eb-4cd6-b0b8-6afafd5af146 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:42.075189 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 884/1809] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:32:41 2022] DELETE /identity/v3/projects/700bbb441eec410ca0f39ba8854cbbb2 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:42.078442 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37171794-ecad-4e03-8a40-5c6d2d9f34cb None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:42.079566 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37171794-ecad-4e03-8a40-5c6d2d9f34cb None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:42.080080 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37171794-ecad-4e03-8a40-5c6d2d9f34cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:42.107196 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9d98beda-53ea-4315-9803-d6f51682fb01 admin admin] The token cache is being invalidate because project fddb2c9bfa0e4d9c9cd2bdf2b28b1616 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:32:42.107490 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9d98beda-53ea-4315-9803-d6f51682fb01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:42.108195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9d98beda-53ea-4315-9803-d6f51682fb01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:42.113344 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 926/1810] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:32:41 2022] DELETE /identity/v3/projects/fddb2c9bfa0e4d9c9cd2bdf2b28b1616 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:42.113344 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d98beda-53ea-4315-9803-d6f51682fb01 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:42.120793 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9d98beda-53ea-4315-9803-d6f51682fb01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:42.122010 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-37171794-ecad-4e03-8a40-5c6d2d9f34cb None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:42.123674 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-37171794-ecad-4e03-8a40-5c6d2d9f34cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:42.253393 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 885/1811] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:42.254630 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db676ca-4d1f-4923-a148-3ca09817a2c3 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:42.256015 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db676ca-4d1f-4923-a148-3ca09817a2c3 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:42.256685 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db676ca-4d1f-4923-a148-3ca09817a2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:42.279757 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d98beda-53ea-4315-9803-d6f51682fb01 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:42.286284 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-8db676ca-4d1f-4923-a148-3ca09817a2c3 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:42.287853 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8db676ca-4d1f-4923-a148-3ca09817a2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:42.290409 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9273b4-4700-41d3-b788-5e1c738e1bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:42.291339 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9273b4-4700-41d3-b788-5e1c738e1bbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:42.291627 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9273b4-4700-41d3-b788-5e1c738e1bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:42.291906 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9273b4-4700-41d3-b788-5e1c738e1bbf admin admin] PATH_INFO: `/v3/users/34255f546ba34801aeeb33e2677b1e6a` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:42.293262 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9273b4-4700-41d3-b788-5e1c738e1bbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=34255f546ba34801aeeb33e2677b1e6a)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:42.295473 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9273b4-4700-41d3-b788-5e1c738e1bbf admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:42.298685 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 886/1812] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:42.300088 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db676ca-4d1f-4923-a148-3ca09817a2c3 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:42.320123 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db676ca-4d1f-4923-a148-3ca09817a2c3 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:42.326044 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4891ad0-9180-48e7-a9c7-86e9dc186058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:42.326926 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4891ad0-9180-48e7-a9c7-86e9dc186058 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:42.327499 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4891ad0-9180-48e7-a9c7-86e9dc186058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:42.327779 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4891ad0-9180-48e7-a9c7-86e9dc186058 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:42.329907 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4891ad0-9180-48e7-a9c7-86e9dc186058 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:32:42.330116 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4891ad0-9180-48e7-a9c7-86e9dc186058 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:42.333529 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4891ad0-9180-48e7-a9c7-86e9dc186058 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:42.358888 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 887/1813] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:32:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:32:42.360554 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f386c-4c42-4b0c-b61f-f79ef93b65f1 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:42.360863 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f386c-4c42-4b0c-b61f-f79ef93b65f1 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:42.361128 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f386c-4c42-4b0c-b61f-f79ef93b65f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:42.391385 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3b9273b4-4700-41d3-b788-5e1c738e1bbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34255f546ba34801aeeb33e2677b1e6a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:42.403340 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-0e0f386c-4c42-4b0c-b61f-f79ef93b65f1 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:42.406919 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0e0f386c-4c42-4b0c-b61f-f79ef93b65f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:42.423407 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3b9273b4-4700-41d3-b788-5e1c738e1bbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34255f546ba34801aeeb33e2677b1e6a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:42.424205 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3b9273b4-4700-41d3-b788-5e1c738e1bbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34255f546ba34801aeeb33e2677b1e6a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:42.426118 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3b9273b4-4700-41d3-b788-5e1c738e1bbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34255f546ba34801aeeb33e2677b1e6a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:42.433651 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 888/1814] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:42.433651 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f386c-4c42-4b0c-b61f-f79ef93b65f1 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:42.445310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3b9273b4-4700-41d3-b788-5e1c738e1bbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34255f546ba34801aeeb33e2677b1e6a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:42.451565 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f386c-4c42-4b0c-b61f-f79ef93b65f1 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:42.459132 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a98e0-1c17-495a-ac7e-8732a886d625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:42.459930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a98e0-1c17-495a-ac7e-8732a886d625 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:42.460210 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a98e0-1c17-495a-ac7e-8732a886d625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:42.460485 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a98e0-1c17-495a-ac7e-8732a886d625 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:42.461604 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0a98e0-1c17-495a-ac7e-8732a886d625 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:42.464356 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0a98e0-1c17-495a-ac7e-8732a886d625 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:42.471655 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 927/1815] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:32:42 2022] DELETE /identity/v3/users/34255f546ba34801aeeb33e2677b1e6a => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:42.473157 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9273b4-4700-41d3-b788-5e1c738e1bbf admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:42.478029 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3b9273b4-4700-41d3-b788-5e1c738e1bbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:42.524119 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 889/1816] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:32:42 2022] POST /identity/v3/projects => generated 385 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:42.528496 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:42.530077 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a98e0-1c17-495a-ac7e-8732a886d625 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:42.535209 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-fd0a98e0-1c17-495a-ac7e-8732a886d625 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:42.628722 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9273b4-4700-41d3-b788-5e1c738e1bbf admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:42.695471 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a98e0-1c17-495a-ac7e-8732a886d625 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:42.702703 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a40d0f4-1cac-46f8-bf17-e204fca435e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:42.703636 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a40d0f4-1cac-46f8-bf17-e204fca435e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:42.704688 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a40d0f4-1cac-46f8-bf17-e204fca435e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:42.704974 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a40d0f4-1cac-46f8-bf17-e204fca435e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:42.705834 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a40d0f4-1cac-46f8-bf17-e204fca435e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:42.710377 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a40d0f4-1cac-46f8-bf17-e204fca435e8 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:42.754945 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:42.768233 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 890/1817] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:32:42 2022] POST /identity/v3/users => generated 293 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:42.785501 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a40d0f4-1cac-46f8-bf17-e204fca435e8 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:42.786129 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a7e5e6-cb8d-44db-9169-e47349d5c7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:42.787961 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a7e5e6-cb8d-44db-9169-e47349d5c7e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:42.788252 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a7e5e6-cb8d-44db-9169-e47349d5c7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:42.789951 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a7e5e6-cb8d-44db-9169-e47349d5c7e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:42.791975 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a7e5e6-cb8d-44db-9169-e47349d5c7e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:32:42.792843 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a7e5e6-cb8d-44db-9169-e47349d5c7e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:42.794578 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a7e5e6-cb8d-44db-9169-e47349d5c7e3 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:42.808187 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a40d0f4-1cac-46f8-bf17-e204fca435e8 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:42.823637 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 928/1818] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:32:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:32:42.830109 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f445e8f-f0bd-4f3f-9272-f4696efdd80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:42.830885 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f445e8f-f0bd-4f3f-9272-f4696efdd80f admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:42.831207 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f445e8f-f0bd-4f3f-9272-f4696efdd80f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:42.831487 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f445e8f-f0bd-4f3f-9272-f4696efdd80f admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:42.832245 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f445e8f-f0bd-4f3f-9272-f4696efdd80f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:42.833937 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f445e8f-f0bd-4f3f-9272-f4696efdd80f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:42.840863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a7e5e6-cb8d-44db-9169-e47349d5c7e3 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:42.856929 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a7e5e6-cb8d-44db-9169-e47349d5c7e3 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:42.863807 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 891/1819] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:42.864418 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f445e8f-f0bd-4f3f-9272-f4696efdd80f admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:42.869757 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327da31-1a78-4bd9-aea6-c20dad261b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:42.870677 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327da31-1a78-4bd9-aea6-c20dad261b03 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:42.871051 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327da31-1a78-4bd9-aea6-c20dad261b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:42.872218 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327da31-1a78-4bd9-aea6-c20dad261b03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:42.872823 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1327da31-1a78-4bd9-aea6-c20dad261b03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:42.875069 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1327da31-1a78-4bd9-aea6-c20dad261b03 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:42.875264 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f445e8f-f0bd-4f3f-9272-f4696efdd80f admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:42.888280 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f610859-3909-4a4e-ba46-520999651fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:42.893896 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f610859-3909-4a4e-ba46-520999651fe4 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:42.894468 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f610859-3909-4a4e-ba46-520999651fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:42.894909 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f610859-3909-4a4e-ba46-520999651fe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:42.896107 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f610859-3909-4a4e-ba46-520999651fe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:42.898699 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f610859-3909-4a4e-ba46-520999651fe4 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:42.923963 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 892/1821] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:42.924666 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 929/1821] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:32:42 2022] POST /identity/v3/projects => generated 379 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:42.925494 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f610859-3909-4a4e-ba46-520999651fe4 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:42.927459 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327da31-1a78-4bd9-aea6-c20dad261b03 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:42.963275 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f610859-3909-4a4e-ba46-520999651fe4 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:43.070118 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327da31-1a78-4bd9-aea6-c20dad261b03 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:43.084589 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527512ee-62f6-4a7a-afc0-d90825e0a33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:43.090320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527512ee-62f6-4a7a-afc0-d90825e0a33a admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:43.090652 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927af56-e46b-4a0c-a0d9-bb48364e86cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:43.090760 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527512ee-62f6-4a7a-afc0-d90825e0a33a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:43.090760 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527512ee-62f6-4a7a-afc0-d90825e0a33a admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:43.090906 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927af56-e46b-4a0c-a0d9-bb48364e86cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:43.091733 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527512ee-62f6-4a7a-afc0-d90825e0a33a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:43.092087 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927af56-e46b-4a0c-a0d9-bb48364e86cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:43.092929 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927af56-e46b-4a0c-a0d9-bb48364e86cc admin admin] PATH_INFO: `/v3/projects/0f1261c2d43145e1b55b8f5d882eabb6/users/38fcd26af24f49cab7c231db4245b770/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:43.099849 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527512ee-62f6-4a7a-afc0-d90825e0a33a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:43.155424 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2927af56-e46b-4a0c-a0d9-bb48364e86cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f1261c2d43145e1b55b8f5d882eabb6, user_id=38fcd26af24f49cab7c231db4245b770, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:43.157884 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2927af56-e46b-4a0c-a0d9-bb48364e86cc admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:43.166608 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 893/1822] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:32:42 2022] POST /identity/v3/users => generated 290 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:43.166608 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527512ee-62f6-4a7a-afc0-d90825e0a33a admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:43.171580 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-527512ee-62f6-4a7a-afc0-d90825e0a33a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:43.178749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 930/1823] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:42 2022] PUT /identity/v3/projects/0f1261c2d43145e1b55b8f5d882eabb6/users/38fcd26af24f49cab7c231db4245b770/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:32:43.179703 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927af56-e46b-4a0c-a0d9-bb48364e86cc admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:43.211761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527512ee-62f6-4a7a-afc0-d90825e0a33a admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:43.235255 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:43.313935 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927af56-e46b-4a0c-a0d9-bb48364e86cc admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:43.317715 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae1949e-a015-49da-813e-9efe8b8f41e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:43.318615 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1949e-a015-49da-813e-9efe8b8f41e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:43.318903 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1949e-a015-49da-813e-9efe8b8f41e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:43.319344 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1949e-a015-49da-813e-9efe8b8f41e1 admin admin] PATH_INFO: `/v3/projects/ce0653e5f17c46aca953f344126c0e91` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:43.320700 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae1949e-a015-49da-813e-9efe8b8f41e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce0653e5f17c46aca953f344126c0e91)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:43.324081 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae1949e-a015-49da-813e-9efe8b8f41e1 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:43.326032 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b966176-d030-4690-a1ff-4f61124df4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:43.326916 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b966176-d030-4690-a1ff-4f61124df4c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:43.327343 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b966176-d030-4690-a1ff-4f61124df4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:43.327719 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b966176-d030-4690-a1ff-4f61124df4c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:43.328569 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b966176-d030-4690-a1ff-4f61124df4c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:43.335130 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b966176-d030-4690-a1ff-4f61124df4c6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:43.356433 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 931/1824] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:43.358055 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ddcfd-80ab-435a-9d39-c989c8600b85 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:43.361467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ddcfd-80ab-435a-9d39-c989c8600b85 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:43.361892 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ddcfd-80ab-435a-9d39-c989c8600b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:43.403383 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bae1949e-a015-49da-813e-9efe8b8f41e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce0653e5f17c46aca953f344126c0e91', 'request_id': 'req-bae1949e-a015-49da-813e-9efe8b8f41e1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:43.419241 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bae1949e-a015-49da-813e-9efe8b8f41e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce0653e5f17c46aca953f344126c0e91', 'request_id': 'req-bae1949e-a015-49da-813e-9efe8b8f41e1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:43.429970 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bae1949e-a015-49da-813e-9efe8b8f41e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce0653e5f17c46aca953f344126c0e91', 'request_id': 'req-bae1949e-a015-49da-813e-9efe8b8f41e1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:43.430961 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a16ddcfd-80ab-435a-9d39-c989c8600b85 None None] MFA Rules not processed for user `38fcd26af24f49cab7c231db4245b770`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:43.432722 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a16ddcfd-80ab-435a-9d39-c989c8600b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:43.437616 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:43.450077 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bae1949e-a015-49da-813e-9efe8b8f41e1 admin admin] The token cache is being invalidate because project ce0653e5f17c46aca953f344126c0e91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:32:43.450692 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bae1949e-a015-49da-813e-9efe8b8f41e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:43.451452 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bae1949e-a015-49da-813e-9efe8b8f41e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:43.455214 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 894/1825] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:32:43 2022] DELETE /identity/v3/projects/ce0653e5f17c46aca953f344126c0e91 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:43.456610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae1949e-a015-49da-813e-9efe8b8f41e1 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:43.461524 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-bae1949e-a015-49da-813e-9efe8b8f41e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:43.521334 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 932/1826] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:43.522649 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ddcfd-80ab-435a-9d39-c989c8600b85 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:43.526252 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a16ddcfd-80ab-435a-9d39-c989c8600b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:43.564778 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae1949e-a015-49da-813e-9efe8b8f41e1 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:43.571385 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec45162-fbb3-49b8-a80c-36054ed11eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:43.572345 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec45162-fbb3-49b8-a80c-36054ed11eca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:43.572801 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec45162-fbb3-49b8-a80c-36054ed11eca admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:43.573085 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec45162-fbb3-49b8-a80c-36054ed11eca admin admin] PATH_INFO: `/v3/users/d87dc63cdfc44556b92138b85a4dd49f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:43.574559 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec45162-fbb3-49b8-a80c-36054ed11eca admin admin] RBAC: Authorizing `identity:delete_user(user_id=d87dc63cdfc44556b92138b85a4dd49f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:43.576506 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec45162-fbb3-49b8-a80c-36054ed11eca admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:43.671330 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2ec45162-fbb3-49b8-a80c-36054ed11eca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd87dc63cdfc44556b92138b85a4dd49f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:43.672275 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2ec45162-fbb3-49b8-a80c-36054ed11eca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd87dc63cdfc44556b92138b85a4dd49f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:43.707006 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2ec45162-fbb3-49b8-a80c-36054ed11eca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd87dc63cdfc44556b92138b85a4dd49f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:43.707991 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ddcfd-80ab-435a-9d39-c989c8600b85 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:43.722678 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2ec45162-fbb3-49b8-a80c-36054ed11eca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd87dc63cdfc44556b92138b85a4dd49f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:43.724497 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1320cf-e8a9-4a19-80b1-b378c5a7f503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:43.725209 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1320cf-e8a9-4a19-80b1-b378c5a7f503 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:43.726365 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1320cf-e8a9-4a19-80b1-b378c5a7f503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:43.726646 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1320cf-e8a9-4a19-80b1-b378c5a7f503 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:43.730828 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1320cf-e8a9-4a19-80b1-b378c5a7f503 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:43.734922 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1320cf-e8a9-4a19-80b1-b378c5a7f503 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:43.738060 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2ec45162-fbb3-49b8-a80c-36054ed11eca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd87dc63cdfc44556b92138b85a4dd49f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:43.749874 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 933/1827] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:43.751430 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73020b1e-4582-401f-b423-742ea17418e5 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:43.751722 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73020b1e-4582-401f-b423-742ea17418e5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:43.752001 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73020b1e-4582-401f-b423-742ea17418e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:43.769060 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 895/1828] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:32:43 2022] DELETE /identity/v3/users/d87dc63cdfc44556b92138b85a4dd49f => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:43.770645 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec45162-fbb3-49b8-a80c-36054ed11eca admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:43.774468 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-2ec45162-fbb3-49b8-a80c-36054ed11eca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:43.775218 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-73020b1e-4582-401f-b423-742ea17418e5 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:43.776588 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-73020b1e-4582-401f-b423-742ea17418e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:43.886620 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:43.941557 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 934/1829] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:43.943146 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73020b1e-4582-401f-b423-742ea17418e5 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:43.944883 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-73020b1e-4582-401f-b423-742ea17418e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:43.958859 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73020b1e-4582-401f-b423-742ea17418e5 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:43.964499 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c6093c-61dc-4954-b687-71cb738a6396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:43.966055 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6093c-61dc-4954-b687-71cb738a6396 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:43.966389 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6093c-61dc-4954-b687-71cb738a6396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:43.966669 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6093c-61dc-4954-b687-71cb738a6396 admin admin] PATH_INFO: `/v3/projects/2a2c700ac5bd4095a01d35a66a4596a3/users/adee96124b7b4384bc89197328a9dc2d/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:44.004619 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c6093c-61dc-4954-b687-71cb738a6396 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a2c700ac5bd4095a01d35a66a4596a3, user_id=adee96124b7b4384bc89197328a9dc2d, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:44.010387 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c6093c-61dc-4954-b687-71cb738a6396 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:44.040876 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 935/1830] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:43 2022] PUT /identity/v3/projects/2a2c700ac5bd4095a01d35a66a4596a3/users/adee96124b7b4384bc89197328a9dc2d/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:44.041844 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec45162-fbb3-49b8-a80c-36054ed11eca admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:44.042655 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8100a2b-b6fd-48e4-a23e-6eedf99b44af None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:44.042976 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8100a2b-b6fd-48e4-a23e-6eedf99b44af None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:44.043405 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8100a2b-b6fd-48e4-a23e-6eedf99b44af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:44.068804 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e8100a2b-b6fd-48e4-a23e-6eedf99b44af None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:44.071494 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e8100a2b-b6fd-48e4-a23e-6eedf99b44af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:44.073719 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:44.161982 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d868a-11dd-4a0a-979b-bbb4032bd88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:44.163724 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d868a-11dd-4a0a-979b-bbb4032bd88d admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:44.164116 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d868a-11dd-4a0a-979b-bbb4032bd88d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:44.164497 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d868a-11dd-4a0a-979b-bbb4032bd88d admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:44.165323 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743d868a-11dd-4a0a-979b-bbb4032bd88d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:44.167906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743d868a-11dd-4a0a-979b-bbb4032bd88d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:44.232940 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 936/1831] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:44.234316 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a11d3e8-4caf-4b82-8994-d316e2416278 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:44.234612 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a11d3e8-4caf-4b82-8994-d316e2416278 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:44.234938 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a11d3e8-4caf-4b82-8994-d316e2416278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:44.247632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 896/1832] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:32:43 2022] POST /identity/v3/projects => generated 387 bytes in 478 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:44.251623 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee95c8-deac-4f99-ae30-5b9a0666bec6 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:44.252101 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee95c8-deac-4f99-ae30-5b9a0666bec6 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:44.252709 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee95c8-deac-4f99-ae30-5b9a0666bec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:44.276883 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9a11d3e8-4caf-4b82-8994-d316e2416278 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:44.280914 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9a11d3e8-4caf-4b82-8994-d316e2416278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:44.301776 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-faee95c8-deac-4f99-ae30-5b9a0666bec6 None None] MFA Rules not processed for user `adee96124b7b4384bc89197328a9dc2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:44.305528 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-faee95c8-deac-4f99-ae30-5b9a0666bec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:44.388201 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 937/1833] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:44.389065 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a11d3e8-4caf-4b82-8994-d316e2416278 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:44.417952 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 897/1834] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:44.417952 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee95c8-deac-4f99-ae30-5b9a0666bec6 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:44.418603 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a11d3e8-4caf-4b82-8994-d316e2416278 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:44.420318 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-faee95c8-deac-4f99-ae30-5b9a0666bec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:44.429881 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfec358-d55a-415d-9550-b2f076628280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:44.430626 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfec358-d55a-415d-9550-b2f076628280 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:44.430918 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfec358-d55a-415d-9550-b2f076628280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:44.431194 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfec358-d55a-415d-9550-b2f076628280 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:44.431909 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfec358-d55a-415d-9550-b2f076628280 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:44.433811 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfec358-d55a-415d-9550-b2f076628280 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:44.464083 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 938/1835] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:32:44 2022] POST /identity/v3/users => generated 294 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:44.466068 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f89b99-4aaa-4fd0-8075-1cf15d317e00 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:44.466787 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f89b99-4aaa-4fd0-8075-1cf15d317e00 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:44.467538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f89b99-4aaa-4fd0-8075-1cf15d317e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:44.489118 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a1f89b99-4aaa-4fd0-8075-1cf15d317e00 None None] MFA Rules not processed for user `6d16feb169ac46f7983a6b8045de77c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:44.490546 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a1f89b99-4aaa-4fd0-8075-1cf15d317e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:44.501745 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee95c8-deac-4f99-ae30-5b9a0666bec6 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:44.517118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5b06f-698a-4e4d-a6bd-5f1875971aef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:44.517824 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5b06f-698a-4e4d-a6bd-5f1875971aef service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:44.518113 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5b06f-698a-4e4d-a6bd-5f1875971aef service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:44.518424 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5b06f-698a-4e4d-a6bd-5f1875971aef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:44.528931 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac5b06f-698a-4e4d-a6bd-5f1875971aef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:44.530422 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac5b06f-698a-4e4d-a6bd-5f1875971aef service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:44.536671 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:44.543665 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 898/1836] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:44.544560 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5b06f-698a-4e4d-a6bd-5f1875971aef service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:44.556339 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5b06f-698a-4e4d-a6bd-5f1875971aef service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:44.559670 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 939/1837] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:44.560543 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f89b99-4aaa-4fd0-8075-1cf15d317e00 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:44.567490 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cbf2f5-9956-4018-95a5-36a4e9bb3da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:44.568365 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cbf2f5-9956-4018-95a5-36a4e9bb3da7 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:44.569334 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cbf2f5-9956-4018-95a5-36a4e9bb3da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:44.569954 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cbf2f5-9956-4018-95a5-36a4e9bb3da7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:44.571125 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cbf2f5-9956-4018-95a5-36a4e9bb3da7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:32:44.571806 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cbf2f5-9956-4018-95a5-36a4e9bb3da7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:44.574847 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cbf2f5-9956-4018-95a5-36a4e9bb3da7 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:44.577764 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f89b99-4aaa-4fd0-8075-1cf15d317e00 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:44.583507 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f5c41d-03b7-454a-ad91-bcce6a1a865f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:44.584473 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5c41d-03b7-454a-ad91-bcce6a1a865f admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:44.584964 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5c41d-03b7-454a-ad91-bcce6a1a865f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:44.585314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5c41d-03b7-454a-ad91-bcce6a1a865f admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:44.586047 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f5c41d-03b7-454a-ad91-bcce6a1a865f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:44.588066 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f5c41d-03b7-454a-ad91-bcce6a1a865f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:44.598859 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 899/1838] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:32:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:32:44.598859 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cbf2f5-9956-4018-95a5-36a4e9bb3da7 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:44.613042 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cbf2f5-9956-4018-95a5-36a4e9bb3da7 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:44.622584 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650e285-8e89-4e25-8086-61c19d1ef95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:44.624797 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650e285-8e89-4e25-8086-61c19d1ef95a admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:44.625121 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650e285-8e89-4e25-8086-61c19d1ef95a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:44.625401 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650e285-8e89-4e25-8086-61c19d1ef95a admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:44.626137 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5650e285-8e89-4e25-8086-61c19d1ef95a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:44.627802 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5650e285-8e89-4e25-8086-61c19d1ef95a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:44.634823 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 940/1839] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:32:44 2022] POST /identity/v3/projects => generated 379 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:44.643313 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f5c41d-03b7-454a-ad91-bcce6a1a865f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:44.650468 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 900/1840] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:44.651731 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650e285-8e89-4e25-8086-61c19d1ef95a admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:44.666134 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650e285-8e89-4e25-8086-61c19d1ef95a admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:44.683861 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:44.739583 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f5c41d-03b7-454a-ad91-bcce6a1a865f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:44.745938 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aacb0f-b27a-479f-bae3-76f10a92c622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:44.746836 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aacb0f-b27a-479f-bae3-76f10a92c622 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:44.747204 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aacb0f-b27a-479f-bae3-76f10a92c622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:44.747415 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aacb0f-b27a-479f-bae3-76f10a92c622 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:44.748125 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aacb0f-b27a-479f-bae3-76f10a92c622 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:44.750063 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aacb0f-b27a-479f-bae3-76f10a92c622 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:44.756605 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1a0786-4830-45cf-a8e4-21573f9d5f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:44.757866 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a0786-4830-45cf-a8e4-21573f9d5f18 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:44.757866 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a0786-4830-45cf-a8e4-21573f9d5f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:44.757866 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a0786-4830-45cf-a8e4-21573f9d5f18 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:44.758587 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1a0786-4830-45cf-a8e4-21573f9d5f18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:44.761027 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1a0786-4830-45cf-a8e4-21573f9d5f18 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:44.786762 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 941/1841] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:32:44 2022] POST /identity/v3/projects => generated 367 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:44.786972 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 901/1842] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:32:44 2022] POST /identity/v3/users => generated 290 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:44.787656 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1a0786-4830-45cf-a8e4-21573f9d5f18 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:44.787759 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aacb0f-b27a-479f-bae3-76f10a92c622 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:44.800443 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1a0786-4830-45cf-a8e4-21573f9d5f18 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:44.870178 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aacb0f-b27a-479f-bae3-76f10a92c622 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:44.875373 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40321b-860f-4dbe-a66d-9f9b1e17c962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:44.875530 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a37fb-b7c0-4e12-8985-fffe6bbac7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:44.876211 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40321b-860f-4dbe-a66d-9f9b1e17c962 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:44.876342 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a37fb-b7c0-4e12-8985-fffe6bbac7ab admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:44.876506 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40321b-860f-4dbe-a66d-9f9b1e17c962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:44.876603 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a37fb-b7c0-4e12-8985-fffe6bbac7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:44.876783 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40321b-860f-4dbe-a66d-9f9b1e17c962 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:44.876879 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a37fb-b7c0-4e12-8985-fffe6bbac7ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:44.877486 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e40321b-860f-4dbe-a66d-9f9b1e17c962 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:44.877600 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5a37fb-b7c0-4e12-8985-fffe6bbac7ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:44.879320 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5a37fb-b7c0-4e12-8985-fffe6bbac7ab admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:44.879611 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e40321b-860f-4dbe-a66d-9f9b1e17c962 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:44.894172 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 942/1843] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:44.895635 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a37fb-b7c0-4e12-8985-fffe6bbac7ab admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:44.906795 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 902/1844] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:32:44 2022] POST /identity/v3/users => generated 284 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:44.907698 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40321b-860f-4dbe-a66d-9f9b1e17c962 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:44.909381 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a37fb-b7c0-4e12-8985-fffe6bbac7ab admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:44.917140 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dbca8d-38d3-4624-8e79-16cdac703f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:44.917826 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dbca8d-38d3-4624-8e79-16cdac703f6f admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:44.918111 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dbca8d-38d3-4624-8e79-16cdac703f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:44.918538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dbca8d-38d3-4624-8e79-16cdac703f6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:44.918722 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40321b-860f-4dbe-a66d-9f9b1e17c962 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:44.919615 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dbca8d-38d3-4624-8e79-16cdac703f6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:44.921737 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dbca8d-38d3-4624-8e79-16cdac703f6f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:44.923363 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f140b09-8a93-4978-875a-f77e5975c521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:44.924437 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f140b09-8a93-4978-875a-f77e5975c521 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:44.924791 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f140b09-8a93-4978-875a-f77e5975c521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:44.925138 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f140b09-8a93-4978-875a-f77e5975c521 admin admin] PATH_INFO: `/v3/projects/154e8a03d11b47c7b76b1c410d48bc04/users/e5ec697a98874283b2d0d23ba45e40fc/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:44.935474 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 943/1845] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:44.936346 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dbca8d-38d3-4624-8e79-16cdac703f6f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:44.948735 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f140b09-8a93-4978-875a-f77e5975c521 admin admin] RBAC: Authorizing `identity:create_grant(project_id=154e8a03d11b47c7b76b1c410d48bc04, user_id=e5ec697a98874283b2d0d23ba45e40fc, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:44.950071 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dbca8d-38d3-4624-8e79-16cdac703f6f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:44.951715 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f140b09-8a93-4978-875a-f77e5975c521 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:44.955418 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12cfc3d-a244-4e8e-b7e1-5f06854f8715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:44.956359 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12cfc3d-a244-4e8e-b7e1-5f06854f8715 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:44.956856 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12cfc3d-a244-4e8e-b7e1-5f06854f8715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:44.957435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12cfc3d-a244-4e8e-b7e1-5f06854f8715 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:44.958335 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12cfc3d-a244-4e8e-b7e1-5f06854f8715 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:44.961761 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12cfc3d-a244-4e8e-b7e1-5f06854f8715 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:44.977426 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 903/1846] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:44 2022] PUT /identity/v3/projects/154e8a03d11b47c7b76b1c410d48bc04/users/e5ec697a98874283b2d0d23ba45e40fc/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:44.978021 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 944/1847] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:44.982292 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f140b09-8a93-4978-875a-f77e5975c521 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:44.983605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12cfc3d-a244-4e8e-b7e1-5f06854f8715 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:45.000010 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12cfc3d-a244-4e8e-b7e1-5f06854f8715 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:45.032216 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:45.068650 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f140b09-8a93-4978-875a-f77e5975c521 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:45.073407 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa508dcd-065c-44c2-a9e6-500a0d932242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:45.073824 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33523bc4-2a2d-415d-8d59-d1e26489bed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:45.074098 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa508dcd-065c-44c2-a9e6-500a0d932242 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:45.074414 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa508dcd-065c-44c2-a9e6-500a0d932242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:45.074557 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33523bc4-2a2d-415d-8d59-d1e26489bed7 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:45.074686 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa508dcd-065c-44c2-a9e6-500a0d932242 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:45.074812 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33523bc4-2a2d-415d-8d59-d1e26489bed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:45.075094 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33523bc4-2a2d-415d-8d59-d1e26489bed7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:45.075457 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa508dcd-065c-44c2-a9e6-500a0d932242 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:45.075923 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33523bc4-2a2d-415d-8d59-d1e26489bed7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:45.077074 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa508dcd-065c-44c2-a9e6-500a0d932242 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:45.077605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33523bc4-2a2d-415d-8d59-d1e26489bed7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:45.088598 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 904/1848] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:45.089457 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 945/1849] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:45.090399 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33523bc4-2a2d-415d-8d59-d1e26489bed7 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:45.093170 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d783e6-6189-4fcb-b41a-c1366f8445a7 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:45.093450 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d783e6-6189-4fcb-b41a-c1366f8445a7 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:45.093625 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-33523bc4-2a2d-415d-8d59-d1e26489bed7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:45.093799 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d783e6-6189-4fcb-b41a-c1366f8445a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:45.107642 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:45.120463 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33523bc4-2a2d-415d-8d59-d1e26489bed7 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:45.124329 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-57d783e6-6189-4fcb-b41a-c1366f8445a7 None None] MFA Rules not processed for user `e5ec697a98874283b2d0d23ba45e40fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:45.125565 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-57d783e6-6189-4fcb-b41a-c1366f8445a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:45.127069 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c426efa-0f06-4565-876d-a001788774ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:45.128196 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c426efa-0f06-4565-876d-a001788774ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:45.128712 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c426efa-0f06-4565-876d-a001788774ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:45.129061 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c426efa-0f06-4565-876d-a001788774ec admin admin] PATH_INFO: `/v3/projects/799367c091bf4817ae4af6651f3cd47f` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:45.131258 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c426efa-0f06-4565-876d-a001788774ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=799367c091bf4817ae4af6651f3cd47f)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:45.132828 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c426efa-0f06-4565-876d-a001788774ec admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:45.205566 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5c426efa-0f06-4565-876d-a001788774ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '799367c091bf4817ae4af6651f3cd47f', 'request_id': 'req-5c426efa-0f06-4565-876d-a001788774ec'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:45.216300 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5c426efa-0f06-4565-876d-a001788774ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '799367c091bf4817ae4af6651f3cd47f', 'request_id': 'req-5c426efa-0f06-4565-876d-a001788774ec'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:45.224410 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5c426efa-0f06-4565-876d-a001788774ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '799367c091bf4817ae4af6651f3cd47f', 'request_id': 'req-5c426efa-0f06-4565-876d-a001788774ec'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:45.245076 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 905/1850] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:45.246699 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d783e6-6189-4fcb-b41a-c1366f8445a7 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:45.251261 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5c426efa-0f06-4565-876d-a001788774ec admin admin] The token cache is being invalidate because project 799367c091bf4817ae4af6651f3cd47f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:32:45.251808 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5c426efa-0f06-4565-876d-a001788774ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:45.253817 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5c426efa-0f06-4565-876d-a001788774ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:45.255956 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 946/1851] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:32:45 2022] DELETE /identity/v3/projects/799367c091bf4817ae4af6651f3cd47f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:45.257139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c426efa-0f06-4565-876d-a001788774ec admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:45.260690 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5c426efa-0f06-4565-876d-a001788774ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:45.279743 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d783e6-6189-4fcb-b41a-c1366f8445a7 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:45.345853 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c426efa-0f06-4565-876d-a001788774ec admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:45.351539 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aeca98-cc8d-41f2-ade8-77e005597ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:45.352783 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aeca98-cc8d-41f2-ade8-77e005597ee5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:45.353174 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aeca98-cc8d-41f2-ade8-77e005597ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:45.353526 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aeca98-cc8d-41f2-ade8-77e005597ee5 admin admin] PATH_INFO: `/v3/projects/f96a07ebd0b945f9be149bf9fe4bc10a/users/ee2b27e4185248dbab81460f5f80b24d/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:45.372961 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aeca98-cc8d-41f2-ade8-77e005597ee5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f96a07ebd0b945f9be149bf9fe4bc10a, user_id=ee2b27e4185248dbab81460f5f80b24d, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:45.376205 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aeca98-cc8d-41f2-ade8-77e005597ee5 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:45.398382 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 947/1852] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:45 2022] PUT /identity/v3/projects/f96a07ebd0b945f9be149bf9fe4bc10a/users/ee2b27e4185248dbab81460f5f80b24d/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:45.399968 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aeca98-cc8d-41f2-ade8-77e005597ee5 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:45.402632 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b5aeca98-cc8d-41f2-ade8-77e005597ee5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:45.428587 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884c7192-49f7-445a-9441-b481c1dbb11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:45.429673 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c7192-49f7-445a-9441-b481c1dbb11c admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:45.429958 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c7192-49f7-445a-9441-b481c1dbb11c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:45.430230 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c7192-49f7-445a-9441-b481c1dbb11c admin admin] PATH_INFO: `/v3/projects/97b1741fd4b142d7ae8712f2e3488723/users/2b5450ed9550485795cbb63e60a14628/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:45.460304 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884c7192-49f7-445a-9441-b481c1dbb11c admin admin] RBAC: Authorizing `identity:create_grant(project_id=97b1741fd4b142d7ae8712f2e3488723, user_id=2b5450ed9550485795cbb63e60a14628, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:45.463197 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884c7192-49f7-445a-9441-b481c1dbb11c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:45.465718 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:45.504028 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 906/1853] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:45 2022] PUT /identity/v3/projects/97b1741fd4b142d7ae8712f2e3488723/users/2b5450ed9550485795cbb63e60a14628/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:45.504028 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4489f349-578c-4485-85e2-e6723e4ccacf None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:45.504028 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4489f349-578c-4485-85e2-e6723e4ccacf None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:45.504028 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4489f349-578c-4485-85e2-e6723e4ccacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:45.527089 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-4489f349-578c-4485-85e2-e6723e4ccacf None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:45.527941 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4489f349-578c-4485-85e2-e6723e4ccacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:45.550119 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aeca98-cc8d-41f2-ade8-77e005597ee5 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:45.637833 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea51dc-7d81-443f-8dde-d32d68bf1895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:45.638702 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea51dc-7d81-443f-8dde-d32d68bf1895 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:45.638840 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea51dc-7d81-443f-8dde-d32d68bf1895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:45.639119 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea51dc-7d81-443f-8dde-d32d68bf1895 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:45.640779 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea51dc-7d81-443f-8dde-d32d68bf1895 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:45.642817 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea51dc-7d81-443f-8dde-d32d68bf1895 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:45.695794 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 907/1854] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:45.697403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 948/1855] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:32:45 2022] POST /identity/v3/projects => generated 387 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:45.698784 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1e06d-b69a-462a-b74d-4011a93f2ed7 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:45.699078 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1e06d-b69a-462a-b74d-4011a93f2ed7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:45.699362 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1e06d-b69a-462a-b74d-4011a93f2ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:45.701017 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195fdca-228f-49ad-914c-b9648d38cbda None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:45.701306 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195fdca-228f-49ad-914c-b9648d38cbda None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:45.701585 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195fdca-228f-49ad-914c-b9648d38cbda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:45.739091 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-76b1e06d-b69a-462a-b74d-4011a93f2ed7 None None] MFA Rules not processed for user `2b5450ed9550485795cbb63e60a14628`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:45.740327 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-9195fdca-228f-49ad-914c-b9648d38cbda None None] MFA Rules not processed for user `ee2b27e4185248dbab81460f5f80b24d`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:45.740656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-76b1e06d-b69a-462a-b74d-4011a93f2ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:45.741858 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9195fdca-228f-49ad-914c-b9648d38cbda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:45.754182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:45.822736 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 949/1856] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:45.822736 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b1e06d-b69a-462a-b74d-4011a93f2ed7 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:45.881289 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 908/1857] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:45 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:45.883129 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e7d67-e0a4-4173-a163-97a4c03b1551 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:45.883276 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e7d67-e0a4-4173-a163-97a4c03b1551 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:45.883523 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e7d67-e0a4-4173-a163-97a4c03b1551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:45.896682 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:45.913914 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-717e7d67-e0a4-4173-a163-97a4c03b1551 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:45.915208 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-717e7d67-e0a4-4173-a163-97a4c03b1551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:45.919809 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b1e06d-b69a-462a-b74d-4011a93f2ed7 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:45.925690 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26064ed-0dc7-4697-982c-0d468b30028f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:45.926829 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26064ed-0dc7-4697-982c-0d468b30028f admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:45.927301 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26064ed-0dc7-4697-982c-0d468b30028f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:45.927578 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26064ed-0dc7-4697-982c-0d468b30028f admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:45.928288 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26064ed-0dc7-4697-982c-0d468b30028f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:45.930431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26064ed-0dc7-4697-982c-0d468b30028f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:45.934628 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 909/1858] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:45.935989 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717e7d67-e0a4-4173-a163-97a4c03b1551 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:45.937606 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-717e7d67-e0a4-4173-a163-97a4c03b1551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:45.953506 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717e7d67-e0a4-4173-a163-97a4c03b1551 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:45.959344 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664318be-a614-497f-8f72-e0996abd3fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:45.961033 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 950/1859] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:32:45 2022] POST /identity/v3/users => generated 300 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:45.961165 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664318be-a614-497f-8f72-e0996abd3fac admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:45.961244 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664318be-a614-497f-8f72-e0996abd3fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:45.961521 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664318be-a614-497f-8f72-e0996abd3fac admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:45.962222 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26064ed-0dc7-4697-982c-0d468b30028f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:45.962466 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664318be-a614-497f-8f72-e0996abd3fac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:45.964748 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664318be-a614-497f-8f72-e0996abd3fac admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:45.976171 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26064ed-0dc7-4697-982c-0d468b30028f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:45.987246 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3544efa-6e5c-4f4d-9ace-1d6c91881eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:45.988289 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3544efa-6e5c-4f4d-9ace-1d6c91881eb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:45.988289 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3544efa-6e5c-4f4d-9ace-1d6c91881eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:45.988679 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3544efa-6e5c-4f4d-9ace-1d6c91881eb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:45.989480 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3544efa-6e5c-4f4d-9ace-1d6c91881eb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:45.991817 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3544efa-6e5c-4f4d-9ace-1d6c91881eb8 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:46.019976 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 910/1860] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:32:45 2022] POST /identity/v3/projects => generated 379 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:46.019976 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664318be-a614-497f-8f72-e0996abd3fac admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:46.031848 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 951/1861] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:32:45 2022] POST /identity/v3/projects => generated 365 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:46.031848 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3544efa-6e5c-4f4d-9ace-1d6c91881eb8 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:46.127281 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3544efa-6e5c-4f4d-9ace-1d6c91881eb8 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:46.134994 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da854cf9-f3e3-4684-94e8-5ae0b0699b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:46.136049 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da854cf9-f3e3-4684-94e8-5ae0b0699b86 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:46.137013 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da854cf9-f3e3-4684-94e8-5ae0b0699b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:46.137511 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da854cf9-f3e3-4684-94e8-5ae0b0699b86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:46.140572 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da854cf9-f3e3-4684-94e8-5ae0b0699b86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:46.143324 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664318be-a614-497f-8f72-e0996abd3fac admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:46.143437 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da854cf9-f3e3-4684-94e8-5ae0b0699b86 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:46.149223 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16359a-1e0f-470b-9509-9fe6a33c5f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:46.149935 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16359a-1e0f-470b-9509-9fe6a33c5f2b admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:46.150220 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16359a-1e0f-470b-9509-9fe6a33c5f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:46.150523 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16359a-1e0f-470b-9509-9fe6a33c5f2b admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:46.151260 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f16359a-1e0f-470b-9509-9fe6a33c5f2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:32:46.151566 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f16359a-1e0f-470b-9509-9fe6a33c5f2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:46.153181 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f16359a-1e0f-470b-9509-9fe6a33c5f2b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:46.155963 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 952/1862] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:46.156672 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da854cf9-f3e3-4684-94e8-5ae0b0699b86 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:46.170147 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 911/1863] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:32:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:32:46.171256 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16359a-1e0f-470b-9509-9fe6a33c5f2b admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:46.181824 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da854cf9-f3e3-4684-94e8-5ae0b0699b86 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:46.186681 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5a0bd3-7883-49a8-a0de-692d3ccd56ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:46.187486 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a0bd3-7883-49a8-a0de-692d3ccd56ca admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:46.187771 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a0bd3-7883-49a8-a0de-692d3ccd56ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:46.188113 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a0bd3-7883-49a8-a0de-692d3ccd56ca admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:46.188750 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5a0bd3-7883-49a8-a0de-692d3ccd56ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:46.190705 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5a0bd3-7883-49a8-a0de-692d3ccd56ca admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:46.193851 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16359a-1e0f-470b-9509-9fe6a33c5f2b admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:46.198815 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b51ce-1589-4d0a-83af-214ca6965434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:46.199510 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b51ce-1589-4d0a-83af-214ca6965434 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:46.199797 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b51ce-1589-4d0a-83af-214ca6965434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:46.200069 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b51ce-1589-4d0a-83af-214ca6965434 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:46.200777 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b51ce-1589-4d0a-83af-214ca6965434 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:46.202753 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b51ce-1589-4d0a-83af-214ca6965434 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:46.219002 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 953/1864] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:32:46 2022] POST /identity/v3/users => generated 296 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:46.219822 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5a0bd3-7883-49a8-a0de-692d3ccd56ca admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:46.230981 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5a0bd3-7883-49a8-a0de-692d3ccd56ca admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:46.232656 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 912/1865] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:32:46 2022] POST /identity/v3/users => generated 289 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:46.233651 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b51ce-1589-4d0a-83af-214ca6965434 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:46.235871 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb61f5-29d8-45a9-8201-cb2b9d4aedb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:46.236773 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb61f5-29d8-45a9-8201-cb2b9d4aedb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:46.237162 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb61f5-29d8-45a9-8201-cb2b9d4aedb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:46.237458 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb61f5-29d8-45a9-8201-cb2b9d4aedb9 admin admin] PATH_INFO: `/v3/projects/81525ca1a71b40e493bf8f4966909f2c/users/c366065f7bc148eba34fd9bf4c6d19d1/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:46.245231 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b51ce-1589-4d0a-83af-214ca6965434 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:46.250708 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa3576-f5b9-47dc-8579-59077570987a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:46.251305 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa3576-f5b9-47dc-8579-59077570987a admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:46.251584 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa3576-f5b9-47dc-8579-59077570987a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:46.251975 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa3576-f5b9-47dc-8579-59077570987a admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:46.252692 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa3576-f5b9-47dc-8579-59077570987a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:46.254210 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdb61f5-29d8-45a9-8201-cb2b9d4aedb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81525ca1a71b40e493bf8f4966909f2c, user_id=c366065f7bc148eba34fd9bf4c6d19d1, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:46.254994 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa3576-f5b9-47dc-8579-59077570987a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:46.256908 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdb61f5-29d8-45a9-8201-cb2b9d4aedb9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:46.272394 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 954/1866] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:46 2022] PUT /identity/v3/projects/81525ca1a71b40e493bf8f4966909f2c/users/c366065f7bc148eba34fd9bf4c6d19d1/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:46.273232 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb61f5-29d8-45a9-8201-cb2b9d4aedb9 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:46.291041 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 913/1867] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:32:46 2022] POST /identity/v3/projects => generated 403 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:46.291887 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa3576-f5b9-47dc-8579-59077570987a admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:46.340629 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb61f5-29d8-45a9-8201-cb2b9d4aedb9 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:46.349025 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:46.359483 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:46.388449 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa3576-f5b9-47dc-8579-59077570987a admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:46.393426 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd270e-898b-4729-a0e5-705c5ab3b1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:46.394176 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd270e-898b-4729-a0e5-705c5ab3b1c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:46.394452 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd270e-898b-4729-a0e5-705c5ab3b1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:46.394754 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd270e-898b-4729-a0e5-705c5ab3b1c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:46.395472 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd270e-898b-4729-a0e5-705c5ab3b1c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:46.397435 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd270e-898b-4729-a0e5-705c5ab3b1c9 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:46.405598 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b78ad-4b78-40bb-8cb2-cef7c08fa261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:46.406322 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b78ad-4b78-40bb-8cb2-cef7c08fa261 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:46.406606 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b78ad-4b78-40bb-8cb2-cef7c08fa261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:46.406878 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b78ad-4b78-40bb-8cb2-cef7c08fa261 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:46.407621 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b78ad-4b78-40bb-8cb2-cef7c08fa261 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:46.411819 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 914/1868] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:46.411819 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd270e-898b-4729-a0e5-705c5ab3b1c9 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:46.412045 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b78ad-4b78-40bb-8cb2-cef7c08fa261 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:46.420817 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 955/1869] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:46.421472 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b78ad-4b78-40bb-8cb2-cef7c08fa261 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:46.422133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd270e-898b-4729-a0e5-705c5ab3b1c9 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:46.427017 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b74414-5249-43ea-af60-5ccca9e01a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:46.427691 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b74414-5249-43ea-af60-5ccca9e01a58 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:46.427978 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b74414-5249-43ea-af60-5ccca9e01a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:46.428250 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b74414-5249-43ea-af60-5ccca9e01a58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:46.428993 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b74414-5249-43ea-af60-5ccca9e01a58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:46.430620 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b74414-5249-43ea-af60-5ccca9e01a58 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:46.433211 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b78ad-4b78-40bb-8cb2-cef7c08fa261 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:46.438146 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39882c2b-d165-426c-93f4-bbe19a5da65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:46.438895 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39882c2b-d165-426c-93f4-bbe19a5da65c admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:46.439199 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39882c2b-d165-426c-93f4-bbe19a5da65c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:46.439464 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39882c2b-d165-426c-93f4-bbe19a5da65c admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:46.440165 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39882c2b-d165-426c-93f4-bbe19a5da65c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:46.444223 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 915/1870] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:46.444223 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b74414-5249-43ea-af60-5ccca9e01a58 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:46.444441 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39882c2b-d165-426c-93f4-bbe19a5da65c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:46.451890 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b74414-5249-43ea-af60-5ccca9e01a58 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:46.457240 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1289cce4-28c6-416d-a95d-19acbcb8db77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:46.458253 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1289cce4-28c6-416d-a95d-19acbcb8db77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:46.458539 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1289cce4-28c6-416d-a95d-19acbcb8db77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:46.458825 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1289cce4-28c6-416d-a95d-19acbcb8db77 admin admin] PATH_INFO: `/v3/projects/108ee4206ce1487cbd96c16795db8b02/users/8598158783ed4e839b1bd31c220e295c/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:46.467792 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 956/1871] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:32:46 2022] POST /identity/v3/users => generated 302 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:46.467792 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39882c2b-d165-426c-93f4-bbe19a5da65c admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:46.479117 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1289cce4-28c6-416d-a95d-19acbcb8db77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=108ee4206ce1487cbd96c16795db8b02, user_id=8598158783ed4e839b1bd31c220e295c, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:46.482582 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1289cce4-28c6-416d-a95d-19acbcb8db77 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:46.485203 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39882c2b-d165-426c-93f4-bbe19a5da65c admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:46.492757 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64be4a6-c7d6-42a6-90b0-a2777fd74058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:46.496854 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64be4a6-c7d6-42a6-90b0-a2777fd74058 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:46.497180 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64be4a6-c7d6-42a6-90b0-a2777fd74058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:46.497859 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64be4a6-c7d6-42a6-90b0-a2777fd74058 admin admin] PATH_INFO: `/v3/projects/3677baffdaef48d8b2809d56ac5adc33/users/8100966f71e64864b9f1696aaf11507f/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:46.502356 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 916/1872] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:46 2022] PUT /identity/v3/projects/108ee4206ce1487cbd96c16795db8b02/users/8598158783ed4e839b1bd31c220e295c/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:46.503696 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1289cce4-28c6-416d-a95d-19acbcb8db77 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:46.529645 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64be4a6-c7d6-42a6-90b0-a2777fd74058 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3677baffdaef48d8b2809d56ac5adc33, user_id=8100966f71e64864b9f1696aaf11507f, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:46.534921 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64be4a6-c7d6-42a6-90b0-a2777fd74058 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:46.570580 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 957/1873] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:46 2022] PUT /identity/v3/projects/3677baffdaef48d8b2809d56ac5adc33/users/8100966f71e64864b9f1696aaf11507f/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:46.571554 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64be4a6-c7d6-42a6-90b0-a2777fd74058 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:46.604388 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1289cce4-28c6-416d-a95d-19acbcb8db77 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:46.669142 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64be4a6-c7d6-42a6-90b0-a2777fd74058 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:46.674843 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66307d82-9883-428d-bba5-07cec585e6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:46.675618 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66307d82-9883-428d-bba5-07cec585e6c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:46.675901 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66307d82-9883-428d-bba5-07cec585e6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:46.676236 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66307d82-9883-428d-bba5-07cec585e6c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:46.677220 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66307d82-9883-428d-bba5-07cec585e6c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:46.679461 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66307d82-9883-428d-bba5-07cec585e6c2 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:46.682212 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff50803-2fb1-4c76-bcac-2dc6c0649de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:46.683371 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff50803-2fb1-4c76-bcac-2dc6c0649de0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:46.683664 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff50803-2fb1-4c76-bcac-2dc6c0649de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:46.683948 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff50803-2fb1-4c76-bcac-2dc6c0649de0 admin admin] PATH_INFO: `/v3/projects/81525ca1a71b40e493bf8f4966909f2c/users/c366065f7bc148eba34fd9bf4c6d19d1/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:46.686853 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff50803-2fb1-4c76-bcac-2dc6c0649de0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81525ca1a71b40e493bf8f4966909f2c, user_id=c366065f7bc148eba34fd9bf4c6d19d1, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:46.689819 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff50803-2fb1-4c76-bcac-2dc6c0649de0 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:46.701223 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 958/1874] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:46.702187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66307d82-9883-428d-bba5-07cec585e6c2 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:46.703615 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 917/1875] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:46 2022] PUT /identity/v3/projects/81525ca1a71b40e493bf8f4966909f2c/users/c366065f7bc148eba34fd9bf4c6d19d1/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:46.704518 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff50803-2fb1-4c76-bcac-2dc6c0649de0 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:46.714799 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66307d82-9883-428d-bba5-07cec585e6c2 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:46.782396 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:46.819321 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff50803-2fb1-4c76-bcac-2dc6c0649de0 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:46.819732 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e01615-8833-4f4e-803b-526f3438a604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:46.820443 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e01615-8833-4f4e-803b-526f3438a604 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:46.820728 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e01615-8833-4f4e-803b-526f3438a604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:46.821006 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e01615-8833-4f4e-803b-526f3438a604 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:46.821940 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e01615-8833-4f4e-803b-526f3438a604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:46.825146 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e01615-8833-4f4e-803b-526f3438a604 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:46.826714 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec39cfa-26ee-44ea-bae0-a5bccde2e5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:46.827468 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec39cfa-26ee-44ea-bae0-a5bccde2e5a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:46.827764 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec39cfa-26ee-44ea-bae0-a5bccde2e5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:46.828043 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec39cfa-26ee-44ea-bae0-a5bccde2e5a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:46.828783 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec39cfa-26ee-44ea-bae0-a5bccde2e5a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:46.831883 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec39cfa-26ee-44ea-bae0-a5bccde2e5a1 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:46.851337 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 959/1876] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:46.852023 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 918/1877] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:46.853812 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce8143-69db-4443-bd4d-f568b97c172a None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:46.854108 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce8143-69db-4443-bd4d-f568b97c172a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:46.854625 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce8143-69db-4443-bd4d-f568b97c172a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:46.855433 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e01615-8833-4f4e-803b-526f3438a604 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:46.867186 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e01615-8833-4f4e-803b-526f3438a604 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:46.872229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b8ed7-24d4-4acf-a95e-c77f396ccf6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:46.872913 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b8ed7-24d4-4acf-a95e-c77f396ccf6d admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:46.873246 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b8ed7-24d4-4acf-a95e-c77f396ccf6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:46.873481 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b8ed7-24d4-4acf-a95e-c77f396ccf6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:46.874358 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1b8ed7-24d4-4acf-a95e-c77f396ccf6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:46.877001 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1b8ed7-24d4-4acf-a95e-c77f396ccf6d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:46.879128 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:46.895524 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 960/1878] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:46.896872 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b8ed7-24d4-4acf-a95e-c77f396ccf6d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:46.908355 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b8ed7-24d4-4acf-a95e-c77f396ccf6d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:46.912621 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-41ce8143-69db-4443-bd4d-f568b97c172a None None] MFA Rules not processed for user `c366065f7bc148eba34fd9bf4c6d19d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:46.914044 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-41ce8143-69db-4443-bd4d-f568b97c172a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:46.922094 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318647e4-9e49-4fd0-bcd0-baf5d200e713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:46.923522 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318647e4-9e49-4fd0-bcd0-baf5d200e713 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:46.923883 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318647e4-9e49-4fd0-bcd0-baf5d200e713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:46.924235 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318647e4-9e49-4fd0-bcd0-baf5d200e713 admin admin] PATH_INFO: `/v3/projects/108ee4206ce1487cbd96c16795db8b02/users/8598158783ed4e839b1bd31c220e295c/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:46.927070 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318647e4-9e49-4fd0-bcd0-baf5d200e713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=108ee4206ce1487cbd96c16795db8b02, user_id=8598158783ed4e839b1bd31c220e295c, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:46.930291 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318647e4-9e49-4fd0-bcd0-baf5d200e713 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:46.948685 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 961/1879] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:46 2022] PUT /identity/v3/projects/108ee4206ce1487cbd96c16795db8b02/users/8598158783ed4e839b1bd31c220e295c/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:46.949773 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318647e4-9e49-4fd0-bcd0-baf5d200e713 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:47.038807 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 919/1880] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:46 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:47.039755 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ce8143-69db-4443-bd4d-f568b97c172a None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:47.055373 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ce8143-69db-4443-bd4d-f568b97c172a None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:47.059084 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318647e4-9e49-4fd0-bcd0-baf5d200e713 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:47.061563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de2d8a-8bcc-42ff-b7a5-071b49f7a048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:47.062538 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de2d8a-8bcc-42ff-b7a5-071b49f7a048 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:47.062970 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de2d8a-8bcc-42ff-b7a5-071b49f7a048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:47.063400 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de2d8a-8bcc-42ff-b7a5-071b49f7a048 admin admin] PATH_INFO: `/v3/projects/7b36e24252944c789b2d17f45477491b/users/0b6cb0f5395e4b66831e34adb49bcafe/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:47.066612 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245695f0-7526-407b-834f-a01d80365bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:47.067540 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245695f0-7526-407b-834f-a01d80365bae admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:47.067833 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245695f0-7526-407b-834f-a01d80365bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:47.068118 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245695f0-7526-407b-834f-a01d80365bae admin admin] PATH_INFO: `/v3/projects/3677baffdaef48d8b2809d56ac5adc33/users/8100966f71e64864b9f1696aaf11507f/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:47.070260 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245695f0-7526-407b-834f-a01d80365bae admin admin] RBAC: Authorizing `identity:create_grant(project_id=3677baffdaef48d8b2809d56ac5adc33, user_id=8100966f71e64864b9f1696aaf11507f, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:47.074043 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245695f0-7526-407b-834f-a01d80365bae admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:47.086892 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 962/1881] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:46 2022] PUT /identity/v3/projects/3677baffdaef48d8b2809d56ac5adc33/users/8100966f71e64864b9f1696aaf11507f/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:47.088281 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a925cd3e-38e1-4d44-a5e4-e92c2c605e7f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:47.088568 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a925cd3e-38e1-4d44-a5e4-e92c2c605e7f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:47.088858 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a925cd3e-38e1-4d44-a5e4-e92c2c605e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:47.110179 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de2d8a-8bcc-42ff-b7a5-071b49f7a048 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b36e24252944c789b2d17f45477491b, user_id=0b6cb0f5395e4b66831e34adb49bcafe, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:47.113004 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de2d8a-8bcc-42ff-b7a5-071b49f7a048 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:47.137160 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a925cd3e-38e1-4d44-a5e4-e92c2c605e7f None None] MFA Rules not processed for user `8598158783ed4e839b1bd31c220e295c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:47.142945 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a925cd3e-38e1-4d44-a5e4-e92c2c605e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:47.146538 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 920/1882] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:47 2022] PUT /identity/v3/projects/7b36e24252944c789b2d17f45477491b/users/0b6cb0f5395e4b66831e34adb49bcafe/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:47.151415 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de2d8a-8bcc-42ff-b7a5-071b49f7a048 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:47.155285 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-00de2d8a-8bcc-42ff-b7a5-071b49f7a048 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:47.255505 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de2d8a-8bcc-42ff-b7a5-071b49f7a048 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:47.260350 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d3d9fe-cb54-420e-94bc-3c0f541a790f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:47.261383 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d3d9fe-cb54-420e-94bc-3c0f541a790f service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:47.261672 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d3d9fe-cb54-420e-94bc-3c0f541a790f service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:47.261949 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d3d9fe-cb54-420e-94bc-3c0f541a790f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:47.321134 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 963/1883] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:47 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:47.325896 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1cd6f-e28f-4839-8f84-3f03e57ec60b None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:47.326791 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1cd6f-e28f-4839-8f84-3f03e57ec60b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:47.326885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1cd6f-e28f-4839-8f84-3f03e57ec60b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:47.358956 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:47.389046 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d3d9fe-cb54-420e-94bc-3c0f541a790f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:47.390778 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d3d9fe-cb54-420e-94bc-3c0f541a790f service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:47.402159 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0af1cd6f-e28f-4839-8f84-3f03e57ec60b None None] MFA Rules not processed for user `8100966f71e64864b9f1696aaf11507f`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:47.403607 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0af1cd6f-e28f-4839-8f84-3f03e57ec60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:47.421426 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 921/1884] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:47 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:47.426184 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923e6e4-ef2e-4345-a6c9-2a499d95a5a2 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:47.426831 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923e6e4-ef2e-4345-a6c9-2a499d95a5a2 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:47.427317 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923e6e4-ef2e-4345-a6c9-2a499d95a5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:47.480338 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-4923e6e4-ef2e-4345-a6c9-2a499d95a5a2 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:47.484817 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4923e6e4-ef2e-4345-a6c9-2a499d95a5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:47.536557 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:47.613572 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 964/1885] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:47 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:47.613572 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af1cd6f-e28f-4839-8f84-3f03e57ec60b None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:47.613572 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0af1cd6f-e28f-4839-8f84-3f03e57ec60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:47.642863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af1cd6f-e28f-4839-8f84-3f03e57ec60b None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:47.654671 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26a529-9998-4784-adb3-ebd0af3ab406 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:47.655521 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26a529-9998-4784-adb3-ebd0af3ab406 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:47.655808 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26a529-9998-4784-adb3-ebd0af3ab406 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:47.657141 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26a529-9998-4784-adb3-ebd0af3ab406 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:47.681268 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af26a529-9998-4784-adb3-ebd0af3ab406 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:47.684721 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af26a529-9998-4784-adb3-ebd0af3ab406 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:47.708664 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 965/1886] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:47 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:47.804589 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26a529-9998-4784-adb3-ebd0af3ab406 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:47.846136 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26a529-9998-4784-adb3-ebd0af3ab406 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:47.868605 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 922/1887] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 446 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:47.872174 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefb7ee9-7661-43a2-8ff7-d06b08dc18f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:47.876105 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb7ee9-7661-43a2-8ff7-d06b08dc18f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:47.879044 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb7ee9-7661-43a2-8ff7-d06b08dc18f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:47.879894 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb7ee9-7661-43a2-8ff7-d06b08dc18f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:47.919250 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4923e6e4-ef2e-4345-a6c9-2a499d95a5a2 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:47.934703 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4923e6e4-ef2e-4345-a6c9-2a499d95a5a2 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:47.936154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefb7ee9-7661-43a2-8ff7-d06b08dc18f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:47.944877 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72b5f2-08bf-4ab5-8805-3f7f59929361 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:47.945410 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefb7ee9-7661-43a2-8ff7-d06b08dc18f4 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:47.945753 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72b5f2-08bf-4ab5-8805-3f7f59929361 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:47.946042 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72b5f2-08bf-4ab5-8805-3f7f59929361 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:47.946371 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72b5f2-08bf-4ab5-8805-3f7f59929361 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:48.011492 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 966/1888] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:47 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:48.013319 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefb7ee9-7661-43a2-8ff7-d06b08dc18f4 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:48.022955 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fefb7ee9-7661-43a2-8ff7-d06b08dc18f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:48.163619 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e72b5f2-08bf-4ab5-8805-3f7f59929361 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:48.168266 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e72b5f2-08bf-4ab5-8805-3f7f59929361 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:48.219840 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 923/1889] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:47 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:32:48.230699 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288a98d-4e20-407f-8b89-12514088f3ac None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:48.231208 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288a98d-4e20-407f-8b89-12514088f3ac None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:48.232046 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288a98d-4e20-407f-8b89-12514088f3ac None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:48.234198 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 924/1890] 10.222.0.65 () {54 vars in 921 bytes} [Thu May 19 21:32:48 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:32:48.235260 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7288a98d-4e20-407f-8b89-12514088f3ac None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:48.237513 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefb7ee9-7661-43a2-8ff7-d06b08dc18f4 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:48.250669 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7288a98d-4e20-407f-8b89-12514088f3ac None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:48.256847 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a1a353-6595-4fcf-a6fc-7df80ed34a8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:48.257535 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1a353-6595-4fcf-a6fc-7df80ed34a8e service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:48.257775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d7cb2-14b4-4573-ad2c-0b4f960145e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:48.257907 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1a353-6595-4fcf-a6fc-7df80ed34a8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:48.258078 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1a353-6595-4fcf-a6fc-7df80ed34a8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:48.260043 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d7cb2-14b4-4573-ad2c-0b4f960145e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:48.262911 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d7cb2-14b4-4573-ad2c-0b4f960145e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:48.263514 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d7cb2-14b4-4573-ad2c-0b4f960145e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:48.268778 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-fd9d7cb2-14b4-4573-ad2c-0b4f960145e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:48.283134 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a1a353-6595-4fcf-a6fc-7df80ed34a8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:48.285162 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a1a353-6595-4fcf-a6fc-7df80ed34a8e service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:48.308577 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 967/1891] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:48.365860 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d7cb2-14b4-4573-ad2c-0b4f960145e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:48.367763 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d7cb2-14b4-4573-ad2c-0b4f960145e2 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:48.393112 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 925/1892] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:48.434750 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b0737-cc62-42c2-8c61-48bc7b20050e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:48.435045 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b0737-cc62-42c2-8c61-48bc7b20050e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:48.435909 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b0737-cc62-42c2-8c61-48bc7b20050e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:48.469887 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-687b0737-cc62-42c2-8c61-48bc7b20050e None None] MFA Rules not processed for user `e254671b21fe4661a381b9521810cf80`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:48.476304 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-687b0737-cc62-42c2-8c61-48bc7b20050e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:48.633667 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 968/1893] 10.222.0.65 () {58 vars in 1018 bytes} [Thu May 19 21:32:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:48.681616 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d7cb2-14b4-4573-ad2c-0b4f960145e2 service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:48.684222 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-fd9d7cb2-14b4-4573-ad2c-0b4f960145e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:48.722314 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d7cb2-14b4-4573-ad2c-0b4f960145e2 service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:48.739142 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b11706-6919-47a1-b4fd-e180e0c5aa1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:48.740258 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b11706-6919-47a1-b4fd-e180e0c5aa1e service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:48.740699 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b11706-6919-47a1-b4fd-e180e0c5aa1e service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:48.740972 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b11706-6919-47a1-b4fd-e180e0c5aa1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:48.754733 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b11706-6919-47a1-b4fd-e180e0c5aa1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:48.756995 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b11706-6919-47a1-b4fd-e180e0c5aa1e service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:48.772873 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 926/1894] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:49.295124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b0737-cc62-42c2-8c61-48bc7b20050e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:49.313072 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b0737-cc62-42c2-8c61-48bc7b20050e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:49.333589 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1073f9be-91e1-4ca6-bed5-646526d587b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:49.334295 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1073f9be-91e1-4ca6-bed5-646526d587b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:49.334586 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1073f9be-91e1-4ca6-bed5-646526d587b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:49.334868 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1073f9be-91e1-4ca6-bed5-646526d587b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:49.347908 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1073f9be-91e1-4ca6-bed5-646526d587b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:49.510022 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:49.518876 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1073f9be-91e1-4ca6-bed5-646526d587b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:49.521285 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1073f9be-91e1-4ca6-bed5-646526d587b5 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:49.544543 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 969/1895] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:49.576361 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71726a5f-909b-44a6-b28c-a492c9fcc6bd None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:49.576910 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71726a5f-909b-44a6-b28c-a492c9fcc6bd None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:49.576988 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71726a5f-909b-44a6-b28c-a492c9fcc6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:49.619563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:49.647185 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-71726a5f-909b-44a6-b28c-a492c9fcc6bd None None] MFA Rules not processed for user `0b6cb0f5395e4b66831e34adb49bcafe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:49.648410 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-71726a5f-909b-44a6-b28c-a492c9fcc6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:49.760357 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 927/1896] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:49 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:49.761262 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1073f9be-91e1-4ca6-bed5-646526d587b5 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:49.774271 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1073f9be-91e1-4ca6-bed5-646526d587b5 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:49.779946 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cad2d0-eda5-4120-8d83-27ba375ba3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:49.781006 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cad2d0-eda5-4120-8d83-27ba375ba3b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:49.781345 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cad2d0-eda5-4120-8d83-27ba375ba3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:49.781580 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cad2d0-eda5-4120-8d83-27ba375ba3b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:49.782775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cad2d0-eda5-4120-8d83-27ba375ba3b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:49.784641 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cad2d0-eda5-4120-8d83-27ba375ba3b6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:49.822305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 970/1897] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:32:49 2022] POST /identity/v3/projects => generated 405 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:32:49.825146 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71726a5f-909b-44a6-b28c-a492c9fcc6bd None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:49.839846 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71726a5f-909b-44a6-b28c-a492c9fcc6bd None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:49.918730 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf35c0e-9df4-4720-a6af-9ff196dea94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:49.919533 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf35c0e-9df4-4720-a6af-9ff196dea94f admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:49.919766 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf35c0e-9df4-4720-a6af-9ff196dea94f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:49.920041 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf35c0e-9df4-4720-a6af-9ff196dea94f admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:49.920805 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf35c0e-9df4-4720-a6af-9ff196dea94f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:49.922878 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf35c0e-9df4-4720-a6af-9ff196dea94f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:49.973861 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 928/1898] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:32:49 2022] POST /identity/v3/users => generated 309 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:49.975151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cad2d0-eda5-4120-8d83-27ba375ba3b6 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:49.987396 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cad2d0-eda5-4120-8d83-27ba375ba3b6 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:49.993654 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a33b46-dfde-4f77-98c3-47e58de23518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:49.994475 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a33b46-dfde-4f77-98c3-47e58de23518 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:49.994896 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a33b46-dfde-4f77-98c3-47e58de23518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:49.995331 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a33b46-dfde-4f77-98c3-47e58de23518 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:49.996290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a33b46-dfde-4f77-98c3-47e58de23518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:49.998170 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a33b46-dfde-4f77-98c3-47e58de23518 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:50.010718 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 971/1899] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:50.014967 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf35c0e-9df4-4720-a6af-9ff196dea94f admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:50.030416 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf35c0e-9df4-4720-a6af-9ff196dea94f admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:50.034390 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3905ad5-7cca-4b19-ac93-d866740a8fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:50.035531 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3905ad5-7cca-4b19-ac93-d866740a8fb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:50.035863 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3905ad5-7cca-4b19-ac93-d866740a8fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:50.036184 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3905ad5-7cca-4b19-ac93-d866740a8fb0 admin admin] PATH_INFO: `/v3/projects/6cea1b44ce3b4c8dad7f806a0b44047e/users/2b801c7d425049e5af54f5e457864df2/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:50.055369 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3905ad5-7cca-4b19-ac93-d866740a8fb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cea1b44ce3b4c8dad7f806a0b44047e, user_id=2b801c7d425049e5af54f5e457864df2, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:50.058057 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3905ad5-7cca-4b19-ac93-d866740a8fb0 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:50.074737 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 929/1900] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:50 2022] PUT /identity/v3/projects/6cea1b44ce3b4c8dad7f806a0b44047e/users/2b801c7d425049e5af54f5e457864df2/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:50.079202 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a33b46-dfde-4f77-98c3-47e58de23518 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:50.089883 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a33b46-dfde-4f77-98c3-47e58de23518 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:50.168125 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f083443-06de-4fac-b599-e1075186cf62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:50.168807 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f083443-06de-4fac-b599-e1075186cf62 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:50.169081 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f083443-06de-4fac-b599-e1075186cf62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:50.169357 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f083443-06de-4fac-b599-e1075186cf62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:50.170116 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f083443-06de-4fac-b599-e1075186cf62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:50.171926 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f083443-06de-4fac-b599-e1075186cf62 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:50.183342 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 972/1901] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:50.187888 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3905ad5-7cca-4b19-ac93-d866740a8fb0 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:50.203905 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3905ad5-7cca-4b19-ac93-d866740a8fb0 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:50.209627 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d2de5-11ea-4fac-8833-e52687ccd54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:50.210592 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d2de5-11ea-4fac-8833-e52687ccd54e admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:50.210906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d2de5-11ea-4fac-8833-e52687ccd54e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:50.211184 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d2de5-11ea-4fac-8833-e52687ccd54e admin admin] PATH_INFO: `/v3/projects/6cea1b44ce3b4c8dad7f806a0b44047e/users/2b801c7d425049e5af54f5e457864df2/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:50.213332 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d2de5-11ea-4fac-8833-e52687ccd54e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cea1b44ce3b4c8dad7f806a0b44047e, user_id=2b801c7d425049e5af54f5e457864df2, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:50.216093 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d2de5-11ea-4fac-8833-e52687ccd54e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:50.229845 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 930/1902] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:50 2022] PUT /identity/v3/projects/6cea1b44ce3b4c8dad7f806a0b44047e/users/2b801c7d425049e5af54f5e457864df2/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:32:52.187222 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a536e66b-2450-4844-a40c-f1bb186dba81 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:52.187793 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a536e66b-2450-4844-a40c-f1bb186dba81 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:52.188105 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a536e66b-2450-4844-a40c-f1bb186dba81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:52.238849 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a536e66b-2450-4844-a40c-f1bb186dba81 None None] MFA Rules not processed for user `2b801c7d425049e5af54f5e457864df2`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:52.240250 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a536e66b-2450-4844-a40c-f1bb186dba81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:52.424590 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 973/1903] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:52 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:53.052480 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d2de5-11ea-4fac-8833-e52687ccd54e admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:53.154418 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:53.192001 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d2de5-11ea-4fac-8833-e52687ccd54e admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:53.215084 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c080f4b5-37c1-49c4-a042-7aa944f76fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:53.217211 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080f4b5-37c1-49c4-a042-7aa944f76fc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:53.217502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080f4b5-37c1-49c4-a042-7aa944f76fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:53.219950 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080f4b5-37c1-49c4-a042-7aa944f76fc7 admin admin] PATH_INFO: `/v3/users/adee96124b7b4384bc89197328a9dc2d` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:53.222341 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c080f4b5-37c1-49c4-a042-7aa944f76fc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adee96124b7b4384bc89197328a9dc2d)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:53.225578 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c080f4b5-37c1-49c4-a042-7aa944f76fc7 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:53.365645 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c080f4b5-37c1-49c4-a042-7aa944f76fc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adee96124b7b4384bc89197328a9dc2d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:53.366474 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c080f4b5-37c1-49c4-a042-7aa944f76fc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adee96124b7b4384bc89197328a9dc2d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:53.391086 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c080f4b5-37c1-49c4-a042-7aa944f76fc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adee96124b7b4384bc89197328a9dc2d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:53.407882 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c080f4b5-37c1-49c4-a042-7aa944f76fc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adee96124b7b4384bc89197328a9dc2d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:53.426804 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c080f4b5-37c1-49c4-a042-7aa944f76fc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adee96124b7b4384bc89197328a9dc2d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:53.442370 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 931/1904] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:32:53 2022] DELETE /identity/v3/users/adee96124b7b4384bc89197328a9dc2d => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:53.447409 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b45d8-8a90-4321-a85f-a07be06060b0 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:53.449202 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b45d8-8a90-4321-a85f-a07be06060b0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:53.449577 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b45d8-8a90-4321-a85f-a07be06060b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:53.480839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-6c3b45d8-8a90-4321-a85f-a07be06060b0 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:53.482097 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6c3b45d8-8a90-4321-a85f-a07be06060b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:53.531180 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:53.557494 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 974/1905] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:53.572917 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c080f4b5-37c1-49c4-a042-7aa944f76fc7 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:53.574425 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c080f4b5-37c1-49c4-a042-7aa944f76fc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:53.640387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c080f4b5-37c1-49c4-a042-7aa944f76fc7 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:53.645462 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978d117a-eb61-4caa-9ff7-69351deda121 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:53.646485 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d117a-eb61-4caa-9ff7-69351deda121 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:53.646899 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d117a-eb61-4caa-9ff7-69351deda121 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:53.647162 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d117a-eb61-4caa-9ff7-69351deda121 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:53.657992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978d117a-eb61-4caa-9ff7-69351deda121 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:53.659731 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978d117a-eb61-4caa-9ff7-69351deda121 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:53.670812 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 932/1906] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:54.021831 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3b45d8-8a90-4321-a85f-a07be06060b0 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:54.026962 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6c3b45d8-8a90-4321-a85f-a07be06060b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:54.075268 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3b45d8-8a90-4321-a85f-a07be06060b0 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:54.086709 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f8994-e714-41a4-a3da-9638517271e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:54.088367 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f8994-e714-41a4-a3da-9638517271e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:54.092226 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f8994-e714-41a4-a3da-9638517271e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:54.092908 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f8994-e714-41a4-a3da-9638517271e0 admin admin] PATH_INFO: `/v3/projects/2a2c700ac5bd4095a01d35a66a4596a3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:54.105470 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f8994-e714-41a4-a3da-9638517271e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a2c700ac5bd4095a01d35a66a4596a3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:54.109844 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f8994-e714-41a4-a3da-9638517271e0 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:54.216673 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5c9f8994-e714-41a4-a3da-9638517271e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a2c700ac5bd4095a01d35a66a4596a3', 'request_id': 'req-5c9f8994-e714-41a4-a3da-9638517271e0'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:54.236804 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5c9f8994-e714-41a4-a3da-9638517271e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a2c700ac5bd4095a01d35a66a4596a3', 'request_id': 'req-5c9f8994-e714-41a4-a3da-9638517271e0'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:54.246220 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5c9f8994-e714-41a4-a3da-9638517271e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a2c700ac5bd4095a01d35a66a4596a3', 'request_id': 'req-5c9f8994-e714-41a4-a3da-9638517271e0'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:54.265007 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5c9f8994-e714-41a4-a3da-9638517271e0 admin admin] The token cache is being invalidate because project 2a2c700ac5bd4095a01d35a66a4596a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:32:54.265430 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5c9f8994-e714-41a4-a3da-9638517271e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:54.266039 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5c9f8994-e714-41a4-a3da-9638517271e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:54.267938 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 975/1907] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:32:54 2022] DELETE /identity/v3/projects/2a2c700ac5bd4095a01d35a66a4596a3 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:54.272305 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978d117a-eb61-4caa-9ff7-69351deda121 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:54.310922 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978d117a-eb61-4caa-9ff7-69351deda121 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:54.398893 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f8994-e714-41a4-a3da-9638517271e0 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:54.400329 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5c9f8994-e714-41a4-a3da-9638517271e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:54.434774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afe1096-0f90-407f-994c-4be9d750bdcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:54.436870 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe1096-0f90-407f-994c-4be9d750bdcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:54.437738 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe1096-0f90-407f-994c-4be9d750bdcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:54.439391 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe1096-0f90-407f-994c-4be9d750bdcd admin admin] PATH_INFO: `/v3/users/2b5450ed9550485795cbb63e60a14628` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:54.441977 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afe1096-0f90-407f-994c-4be9d750bdcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b5450ed9550485795cbb63e60a14628)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:54.445553 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afe1096-0f90-407f-994c-4be9d750bdcd admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:54.466170 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:54.511643 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f8994-e714-41a4-a3da-9638517271e0 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:54.517776 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263d6e7-cb5d-4388-a038-d2913da91cec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:54.519067 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263d6e7-cb5d-4388-a038-d2913da91cec service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:54.519425 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263d6e7-cb5d-4388-a038-d2913da91cec service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:54.519900 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263d6e7-cb5d-4388-a038-d2913da91cec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:54.520761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7afe1096-0f90-407f-994c-4be9d750bdcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b5450ed9550485795cbb63e60a14628'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:54.521507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7afe1096-0f90-407f-994c-4be9d750bdcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b5450ed9550485795cbb63e60a14628'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:54.522448 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9263d6e7-cb5d-4388-a038-d2913da91cec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:54.540180 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7afe1096-0f90-407f-994c-4be9d750bdcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b5450ed9550485795cbb63e60a14628'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:54.552601 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7afe1096-0f90-407f-994c-4be9d750bdcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b5450ed9550485795cbb63e60a14628'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:54.567895 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7afe1096-0f90-407f-994c-4be9d750bdcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b5450ed9550485795cbb63e60a14628'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:54.587020 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 933/1908] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:32:54 2022] DELETE /identity/v3/users/2b5450ed9550485795cbb63e60a14628 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:54.589515 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53850ea-b477-4184-b7a9-228cd3503525 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:54.589810 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53850ea-b477-4184-b7a9-228cd3503525 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:54.591534 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53850ea-b477-4184-b7a9-228cd3503525 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:54.598340 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 934/1909] 10.222.0.65 () {54 vars in 921 bytes} [Thu May 19 21:32:54 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:32:54.627103 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0e75e-e755-4955-81f8-13d802eb82e4 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:54.627939 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0e75e-e755-4955-81f8-13d802eb82e4 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:54.628496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0e75e-e755-4955-81f8-13d802eb82e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:54.631773 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9263d6e7-cb5d-4388-a038-d2913da91cec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:54.639661 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9263d6e7-cb5d-4388-a038-d2913da91cec service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:54.652625 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-27d0e75e-e755-4955-81f8-13d802eb82e4 None None] MFA Rules not processed for user `f6143bddc05a46ea972709576c7ad95f`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:54.654183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-27d0e75e-e755-4955-81f8-13d802eb82e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:54.657981 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 976/1910] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:54 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:54.825304 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 935/1911] 10.222.0.65 () {58 vars in 1018 bytes} [Thu May 19 21:32:54 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:54.889352 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263d6e7-cb5d-4388-a038-d2913da91cec service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:54.892556 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9263d6e7-cb5d-4388-a038-d2913da91cec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:54.958462 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:55.078928 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263d6e7-cb5d-4388-a038-d2913da91cec service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:55.088163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb6b59-bd9d-4a58-b1a7-6fe819d21b34 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6143bddc05a46ea972709576c7ad95f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:55.090093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb6b59-bd9d-4a58-b1a7-6fe819d21b34 service placement] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:55.090411 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb6b59-bd9d-4a58-b1a7-6fe819d21b34 service placement] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:55.090695 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb6b59-bd9d-4a58-b1a7-6fe819d21b34 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:55.114354 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb6b59-bd9d-4a58-b1a7-6fe819d21b34 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:55.116004 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb6b59-bd9d-4a58-b1a7-6fe819d21b34 service placement] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:55.175603 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 977/1912] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:55.426442 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0e75e-e755-4955-81f8-13d802eb82e4 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:55.431496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-27d0e75e-e755-4955-81f8-13d802eb82e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:55.620329 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0e75e-e755-4955-81f8-13d802eb82e4 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:55.627611 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996aa51d-a547-4ff9-9085-ab6ea9efb0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:55.629026 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996aa51d-a547-4ff9-9085-ab6ea9efb0be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:55.629399 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996aa51d-a547-4ff9-9085-ab6ea9efb0be admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:55.629899 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996aa51d-a547-4ff9-9085-ab6ea9efb0be admin admin] PATH_INFO: `/v3/users/ee2b27e4185248dbab81460f5f80b24d` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:55.632835 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996aa51d-a547-4ff9-9085-ab6ea9efb0be admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee2b27e4185248dbab81460f5f80b24d)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:55.646278 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996aa51d-a547-4ff9-9085-ab6ea9efb0be admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:55.733486 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:55.749492 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb6b59-bd9d-4a58-b1a7-6fe819d21b34 service placement] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:55.750996 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-67cb6b59-bd9d-4a58-b1a7-6fe819d21b34 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:55.768404 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-996aa51d-a547-4ff9-9085-ab6ea9efb0be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee2b27e4185248dbab81460f5f80b24d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:55.769259 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-996aa51d-a547-4ff9-9085-ab6ea9efb0be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee2b27e4185248dbab81460f5f80b24d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:55.771685 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb6b59-bd9d-4a58-b1a7-6fe819d21b34 service placement] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:55.778844 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2391a312-678c-448b-8b1f-2bcd6ab7c7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:55.781675 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2391a312-678c-448b-8b1f-2bcd6ab7c7d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:55.782831 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2391a312-678c-448b-8b1f-2bcd6ab7c7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:55.783186 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2391a312-678c-448b-8b1f-2bcd6ab7c7d8 admin admin] PATH_INFO: `/v3/projects/97b1741fd4b142d7ae8712f2e3488723` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:55.785390 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2391a312-678c-448b-8b1f-2bcd6ab7c7d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97b1741fd4b142d7ae8712f2e3488723)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:55.789488 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2391a312-678c-448b-8b1f-2bcd6ab7c7d8 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:55.801650 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-996aa51d-a547-4ff9-9085-ab6ea9efb0be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee2b27e4185248dbab81460f5f80b24d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:55.822091 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-996aa51d-a547-4ff9-9085-ab6ea9efb0be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee2b27e4185248dbab81460f5f80b24d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:55.853420 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-996aa51d-a547-4ff9-9085-ab6ea9efb0be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee2b27e4185248dbab81460f5f80b24d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:55.869635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 936/1913] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:32:55 2022] DELETE /identity/v3/users/ee2b27e4185248dbab81460f5f80b24d => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:55.874906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596cad1-7292-4139-86a6-2823556dc446 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:55.875210 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596cad1-7292-4139-86a6-2823556dc446 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:55.876563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596cad1-7292-4139-86a6-2823556dc446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:55.912402 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-1596cad1-7292-4139-86a6-2823556dc446 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:55.914061 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1596cad1-7292-4139-86a6-2823556dc446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:55.953086 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2391a312-678c-448b-8b1f-2bcd6ab7c7d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97b1741fd4b142d7ae8712f2e3488723', 'request_id': 'req-2391a312-678c-448b-8b1f-2bcd6ab7c7d8'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:55.977043 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2391a312-678c-448b-8b1f-2bcd6ab7c7d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97b1741fd4b142d7ae8712f2e3488723', 'request_id': 'req-2391a312-678c-448b-8b1f-2bcd6ab7c7d8'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:55.989720 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2391a312-678c-448b-8b1f-2bcd6ab7c7d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97b1741fd4b142d7ae8712f2e3488723', 'request_id': 'req-2391a312-678c-448b-8b1f-2bcd6ab7c7d8'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:56.021678 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2391a312-678c-448b-8b1f-2bcd6ab7c7d8 admin admin] The token cache is being invalidate because project 97b1741fd4b142d7ae8712f2e3488723 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:32:56.022147 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2391a312-678c-448b-8b1f-2bcd6ab7c7d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:56.022985 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2391a312-678c-448b-8b1f-2bcd6ab7c7d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:56.024875 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 978/1914] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:32:55 2022] DELETE /identity/v3/projects/97b1741fd4b142d7ae8712f2e3488723 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:56.029315 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2391a312-678c-448b-8b1f-2bcd6ab7c7d8 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:56.031368 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2391a312-678c-448b-8b1f-2bcd6ab7c7d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:56.066884 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 937/1915] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:56.090272 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1596cad1-7292-4139-86a6-2823556dc446 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:56.092080 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1596cad1-7292-4139-86a6-2823556dc446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:56.162403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2391a312-678c-448b-8b1f-2bcd6ab7c7d8 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:56.177316 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1a687-fdb1-46fc-98ff-a99082f4b55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:56.178468 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1a687-fdb1-46fc-98ff-a99082f4b55b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:56.178948 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1a687-fdb1-46fc-98ff-a99082f4b55b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:56.180265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1a687-fdb1-46fc-98ff-a99082f4b55b admin admin] PATH_INFO: `/v3/users/8100966f71e64864b9f1696aaf11507f` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:56.184476 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a1a687-fdb1-46fc-98ff-a99082f4b55b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8100966f71e64864b9f1696aaf11507f)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:56.186522 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a1a687-fdb1-46fc-98ff-a99082f4b55b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:56.256914 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1596cad1-7292-4139-86a6-2823556dc446 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:56.268619 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb95a0-4b43-475c-b07d-b25056ae85af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:56.269082 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb95a0-4b43-475c-b07d-b25056ae85af service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:56.269368 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb95a0-4b43-475c-b07d-b25056ae85af service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:56.270709 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb95a0-4b43-475c-b07d-b25056ae85af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:56.288540 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb95a0-4b43-475c-b07d-b25056ae85af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:56.297724 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:56.298188 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb95a0-4b43-475c-b07d-b25056ae85af service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:56.318675 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-44a1a687-fdb1-46fc-98ff-a99082f4b55b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8100966f71e64864b9f1696aaf11507f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:56.318893 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 938/1916] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:56.335541 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-44a1a687-fdb1-46fc-98ff-a99082f4b55b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8100966f71e64864b9f1696aaf11507f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:56.336894 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-44a1a687-fdb1-46fc-98ff-a99082f4b55b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8100966f71e64864b9f1696aaf11507f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:56.338451 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-44a1a687-fdb1-46fc-98ff-a99082f4b55b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8100966f71e64864b9f1696aaf11507f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:56.351130 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-44a1a687-fdb1-46fc-98ff-a99082f4b55b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8100966f71e64864b9f1696aaf11507f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:56.378296 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 979/1917] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:32:56 2022] DELETE /identity/v3/users/8100966f71e64864b9f1696aaf11507f => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:56.395252 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe3620-78c2-4846-a19d-46ccf89cef5c None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:56.395535 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe3620-78c2-4846-a19d-46ccf89cef5c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:56.395821 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe3620-78c2-4846-a19d-46ccf89cef5c None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:56.397523 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 939/1918] 10.222.0.65 () {54 vars in 921 bytes} [Thu May 19 21:32:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:32:56.632090 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eea6a0-9e72-4a3e-85c9-007b18d6c1d0 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:56.632896 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eea6a0-9e72-4a3e-85c9-007b18d6c1d0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:56.633438 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eea6a0-9e72-4a3e-85c9-007b18d6c1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:56.658098 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-93eea6a0-9e72-4a3e-85c9-007b18d6c1d0 None None] MFA Rules not processed for user `f6143bddc05a46ea972709576c7ad95f`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:56.659684 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-93eea6a0-9e72-4a3e-85c9-007b18d6c1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:56.727381 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe3620-78c2-4846-a19d-46ccf89cef5c None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:56.729162 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-fafe3620-78c2-4846-a19d-46ccf89cef5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:56.774696 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 980/1919] 10.222.0.65 () {58 vars in 1018 bytes} [Thu May 19 21:32:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:56.776734 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eea6a0-9e72-4a3e-85c9-007b18d6c1d0 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:56.778196 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-93eea6a0-9e72-4a3e-85c9-007b18d6c1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:56.819596 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:56.848297 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe3620-78c2-4846-a19d-46ccf89cef5c None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:56.858817 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7629d92f-7813-4631-838c-44a4af6255e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:56.860965 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629d92f-7813-4631-838c-44a4af6255e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:56.861254 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629d92f-7813-4631-838c-44a4af6255e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:56.861519 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629d92f-7813-4631-838c-44a4af6255e5 admin admin] PATH_INFO: `/v3/projects/3677baffdaef48d8b2809d56ac5adc33` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:56.864164 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7629d92f-7813-4631-838c-44a4af6255e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3677baffdaef48d8b2809d56ac5adc33)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:56.866193 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7629d92f-7813-4631-838c-44a4af6255e5 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:56.874356 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eea6a0-9e72-4a3e-85c9-007b18d6c1d0 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:56.880244 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5582d66a-72e9-4000-a6fa-83f0ce12f4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:56.881275 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582d66a-72e9-4000-a6fa-83f0ce12f4d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:56.881589 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582d66a-72e9-4000-a6fa-83f0ce12f4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:56.881860 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582d66a-72e9-4000-a6fa-83f0ce12f4d2 admin admin] PATH_INFO: `/v3/users/0b6cb0f5395e4b66831e34adb49bcafe` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:56.883146 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5582d66a-72e9-4000-a6fa-83f0ce12f4d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b6cb0f5395e4b66831e34adb49bcafe)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:56.885198 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5582d66a-72e9-4000-a6fa-83f0ce12f4d2 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:56.929825 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7629d92f-7813-4631-838c-44a4af6255e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3677baffdaef48d8b2809d56ac5adc33', 'request_id': 'req-7629d92f-7813-4631-838c-44a4af6255e5'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:56.936853 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7629d92f-7813-4631-838c-44a4af6255e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3677baffdaef48d8b2809d56ac5adc33', 'request_id': 'req-7629d92f-7813-4631-838c-44a4af6255e5'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:56.942911 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5582d66a-72e9-4000-a6fa-83f0ce12f4d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b6cb0f5395e4b66831e34adb49bcafe'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:56.945294 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:56.947158 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7629d92f-7813-4631-838c-44a4af6255e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3677baffdaef48d8b2809d56ac5adc33', 'request_id': 'req-7629d92f-7813-4631-838c-44a4af6255e5'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:56.961079 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7629d92f-7813-4631-838c-44a4af6255e5 admin admin] The token cache is being invalidate because project 3677baffdaef48d8b2809d56ac5adc33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:32:56.962272 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7629d92f-7813-4631-838c-44a4af6255e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:56.963249 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7629d92f-7813-4631-838c-44a4af6255e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:56.963383 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5582d66a-72e9-4000-a6fa-83f0ce12f4d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b6cb0f5395e4b66831e34adb49bcafe'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:56.964118 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5582d66a-72e9-4000-a6fa-83f0ce12f4d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b6cb0f5395e4b66831e34adb49bcafe'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:56.964745 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5582d66a-72e9-4000-a6fa-83f0ce12f4d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b6cb0f5395e4b66831e34adb49bcafe'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:56.965276 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 940/1920] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:32:56 2022] DELETE /identity/v3/projects/3677baffdaef48d8b2809d56ac5adc33 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:56.966352 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7629d92f-7813-4631-838c-44a4af6255e5 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:56.968792 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7629d92f-7813-4631-838c-44a4af6255e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:56.973922 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5582d66a-72e9-4000-a6fa-83f0ce12f4d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b6cb0f5395e4b66831e34adb49bcafe'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:56.993856 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 981/1921] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:32:56 2022] DELETE /identity/v3/users/0b6cb0f5395e4b66831e34adb49bcafe => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:56.995283 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50069153-83a4-4ec5-8b08-9c0bc5cedc2b None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:56.995576 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50069153-83a4-4ec5-8b08-9c0bc5cedc2b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:56.995860 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50069153-83a4-4ec5-8b08-9c0bc5cedc2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:57.022130 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-50069153-83a4-4ec5-8b08-9c0bc5cedc2b None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:57.023468 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-50069153-83a4-4ec5-8b08-9c0bc5cedc2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:57.097182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7629d92f-7813-4631-838c-44a4af6255e5 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:57.132871 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 982/1922] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:57.133936 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50069153-83a4-4ec5-8b08-9c0bc5cedc2b None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:57.141412 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-50069153-83a4-4ec5-8b08-9c0bc5cedc2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:57.157654 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50069153-83a4-4ec5-8b08-9c0bc5cedc2b None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:57.164259 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0bf3b3-3351-44cc-b932-fad6b5c89a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:57.165277 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0bf3b3-3351-44cc-b932-fad6b5c89a98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:57.165664 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0bf3b3-3351-44cc-b932-fad6b5c89a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:57.166041 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0bf3b3-3351-44cc-b932-fad6b5c89a98 admin admin] PATH_INFO: `/v3/projects/f96a07ebd0b945f9be149bf9fe4bc10a` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:57.167425 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0bf3b3-3351-44cc-b932-fad6b5c89a98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f96a07ebd0b945f9be149bf9fe4bc10a)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:57.170005 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0bf3b3-3351-44cc-b932-fad6b5c89a98 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:57.193909 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4566b4-aa53-4ccb-95cf-b3b31b6ed672 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6143bddc05a46ea972709576c7ad95f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:57.194820 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4566b4-aa53-4ccb-95cf-b3b31b6ed672 service placement] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:57.195246 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4566b4-aa53-4ccb-95cf-b3b31b6ed672 service placement] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:57.195664 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4566b4-aa53-4ccb-95cf-b3b31b6ed672 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:57.201949 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0b4566b4-aa53-4ccb-95cf-b3b31b6ed672 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:57.219913 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4566b4-aa53-4ccb-95cf-b3b31b6ed672 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:57.223164 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4566b4-aa53-4ccb-95cf-b3b31b6ed672 service placement] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:57.249149 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ee0bf3b3-3351-44cc-b932-fad6b5c89a98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f96a07ebd0b945f9be149bf9fe4bc10a', 'request_id': 'req-ee0bf3b3-3351-44cc-b932-fad6b5c89a98'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:57.256722 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ee0bf3b3-3351-44cc-b932-fad6b5c89a98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f96a07ebd0b945f9be149bf9fe4bc10a', 'request_id': 'req-ee0bf3b3-3351-44cc-b932-fad6b5c89a98'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:57.264059 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ee0bf3b3-3351-44cc-b932-fad6b5c89a98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f96a07ebd0b945f9be149bf9fe4bc10a', 'request_id': 'req-ee0bf3b3-3351-44cc-b932-fad6b5c89a98'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:57.267430 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 941/1923] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:57.270059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca54aa14-7d2d-4b2e-a945-f2c26d461f91 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:57.273337 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca54aa14-7d2d-4b2e-a945-f2c26d461f91 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:57.274036 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca54aa14-7d2d-4b2e-a945-f2c26d461f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:57.291398 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ee0bf3b3-3351-44cc-b932-fad6b5c89a98 admin admin] The token cache is being invalidate because project f96a07ebd0b945f9be149bf9fe4bc10a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:32:57.292069 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ee0bf3b3-3351-44cc-b932-fad6b5c89a98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:57.293029 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ee0bf3b3-3351-44cc-b932-fad6b5c89a98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:57.299264 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 983/1924] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:32:57 2022] DELETE /identity/v3/projects/f96a07ebd0b945f9be149bf9fe4bc10a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:57.301163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0bf3b3-3351-44cc-b932-fad6b5c89a98 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:57.303293 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ee0bf3b3-3351-44cc-b932-fad6b5c89a98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:57.319082 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-ca54aa14-7d2d-4b2e-a945-f2c26d461f91 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:57.321982 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ca54aa14-7d2d-4b2e-a945-f2c26d461f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:57.350967 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:57.403110 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:57.433404 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 942/1925] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:57.434154 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca54aa14-7d2d-4b2e-a945-f2c26d461f91 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:57.434599 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0bf3b3-3351-44cc-b932-fad6b5c89a98 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:57.439522 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ca54aa14-7d2d-4b2e-a945-f2c26d461f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:57.445391 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a9a13-1798-4364-96ad-3481af29555d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:57.446527 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a9a13-1798-4364-96ad-3481af29555d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:57.446823 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a9a13-1798-4364-96ad-3481af29555d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:57.447179 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a9a13-1798-4364-96ad-3481af29555d admin admin] PATH_INFO: `/v3/users/8598158783ed4e839b1bd31c220e295c` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:57.449355 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a9a13-1798-4364-96ad-3481af29555d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8598158783ed4e839b1bd31c220e295c)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:57.451752 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a9a13-1798-4364-96ad-3481af29555d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:57.452668 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca54aa14-7d2d-4b2e-a945-f2c26d461f91 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:57.458334 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe490a3d-3e6e-4047-bb91-51e402e85584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:57.459129 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe490a3d-3e6e-4047-bb91-51e402e85584 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:57.459410 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe490a3d-3e6e-4047-bb91-51e402e85584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:57.459682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe490a3d-3e6e-4047-bb91-51e402e85584 admin admin] PATH_INFO: `/v3/projects/7b36e24252944c789b2d17f45477491b` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:57.460902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe490a3d-3e6e-4047-bb91-51e402e85584 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b36e24252944c789b2d17f45477491b)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:57.464236 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe490a3d-3e6e-4047-bb91-51e402e85584 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:57.517793 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-234a9a13-1798-4364-96ad-3481af29555d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8598158783ed4e839b1bd31c220e295c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:57.528579 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-fe490a3d-3e6e-4047-bb91-51e402e85584 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b36e24252944c789b2d17f45477491b', 'request_id': 'req-fe490a3d-3e6e-4047-bb91-51e402e85584'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:57.531506 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-234a9a13-1798-4364-96ad-3481af29555d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8598158783ed4e839b1bd31c220e295c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:57.532255 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-234a9a13-1798-4364-96ad-3481af29555d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8598158783ed4e839b1bd31c220e295c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:57.532872 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-234a9a13-1798-4364-96ad-3481af29555d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8598158783ed4e839b1bd31c220e295c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:57.535563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-fe490a3d-3e6e-4047-bb91-51e402e85584 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b36e24252944c789b2d17f45477491b', 'request_id': 'req-fe490a3d-3e6e-4047-bb91-51e402e85584'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:57.541056 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-234a9a13-1798-4364-96ad-3481af29555d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8598158783ed4e839b1bd31c220e295c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:57.546753 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-fe490a3d-3e6e-4047-bb91-51e402e85584 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b36e24252944c789b2d17f45477491b', 'request_id': 'req-fe490a3d-3e6e-4047-bb91-51e402e85584'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:57.555758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 984/1926] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:32:57 2022] DELETE /identity/v3/users/8598158783ed4e839b1bd31c220e295c => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:57.556523 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a9a13-1798-4364-96ad-3481af29555d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:57.558015 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-234a9a13-1798-4364-96ad-3481af29555d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:57.562711 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-fe490a3d-3e6e-4047-bb91-51e402e85584 admin admin] The token cache is being invalidate because project 7b36e24252944c789b2d17f45477491b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:32:57.563123 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-fe490a3d-3e6e-4047-bb91-51e402e85584 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:57.563836 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-fe490a3d-3e6e-4047-bb91-51e402e85584 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:57.565885 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 943/1927] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:32:57 2022] DELETE /identity/v3/projects/7b36e24252944c789b2d17f45477491b => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:57.570078 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe490a3d-3e6e-4047-bb91-51e402e85584 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:57.571653 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-fe490a3d-3e6e-4047-bb91-51e402e85584 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:57.661892 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a9a13-1798-4364-96ad-3481af29555d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:57.668196 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57816c31-a9b5-40df-a10c-cccde61a9965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:57.668894 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57816c31-a9b5-40df-a10c-cccde61a9965 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:57.669181 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57816c31-a9b5-40df-a10c-cccde61a9965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:57.669458 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57816c31-a9b5-40df-a10c-cccde61a9965 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:57.670213 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57816c31-a9b5-40df-a10c-cccde61a9965 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:32:57.670712 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57816c31-a9b5-40df-a10c-cccde61a9965 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:57.672779 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57816c31-a9b5-40df-a10c-cccde61a9965 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:57.684355 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe490a3d-3e6e-4047-bb91-51e402e85584 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:57.692789 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df31c1a-5b6c-496e-b1b2-17fc0b7ed3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:57.694157 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df31c1a-5b6c-496e-b1b2-17fc0b7ed3e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:57.695232 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df31c1a-5b6c-496e-b1b2-17fc0b7ed3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:57.695637 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df31c1a-5b6c-496e-b1b2-17fc0b7ed3e2 admin admin] PATH_INFO: `/v3/users/2b801c7d425049e5af54f5e457864df2` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:57.697307 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 985/1928] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:32:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:32:57.697453 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df31c1a-5b6c-496e-b1b2-17fc0b7ed3e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b801c7d425049e5af54f5e457864df2)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:57.700694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df31c1a-5b6c-496e-b1b2-17fc0b7ed3e2 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:57.701919 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57816c31-a9b5-40df-a10c-cccde61a9965 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:57.705965 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-57816c31-a9b5-40df-a10c-cccde61a9965 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:57.727635 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57816c31-a9b5-40df-a10c-cccde61a9965 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:57.734120 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448276c-0ce2-4aa7-999c-51e855128c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:57.734844 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448276c-0ce2-4aa7-999c-51e855128c5d admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:57.735192 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448276c-0ce2-4aa7-999c-51e855128c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:57.736351 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448276c-0ce2-4aa7-999c-51e855128c5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:57.737174 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4448276c-0ce2-4aa7-999c-51e855128c5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:57.739306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4448276c-0ce2-4aa7-999c-51e855128c5d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:57.780487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 986/1929] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:32:57 2022] POST /identity/v3/projects => generated 375 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:57.781543 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448276c-0ce2-4aa7-999c-51e855128c5d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:57.783379 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4448276c-0ce2-4aa7-999c-51e855128c5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:57.790848 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:57.798420 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-3df31c1a-5b6c-496e-b1b2-17fc0b7ed3e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b801c7d425049e5af54f5e457864df2'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:57.799249 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-3df31c1a-5b6c-496e-b1b2-17fc0b7ed3e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b801c7d425049e5af54f5e457864df2'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:57.815225 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-3df31c1a-5b6c-496e-b1b2-17fc0b7ed3e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b801c7d425049e5af54f5e457864df2'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:57.826502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-3df31c1a-5b6c-496e-b1b2-17fc0b7ed3e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b801c7d425049e5af54f5e457864df2'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:57.837734 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:57.840073 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-3df31c1a-5b6c-496e-b1b2-17fc0b7ed3e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b801c7d425049e5af54f5e457864df2'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:57.854831 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 944/1930] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:32:57 2022] DELETE /identity/v3/users/2b801c7d425049e5af54f5e457864df2 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:57.854831 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df31c1a-5b6c-496e-b1b2-17fc0b7ed3e2 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:57.859811 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3df31c1a-5b6c-496e-b1b2-17fc0b7ed3e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:57.878229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448276c-0ce2-4aa7-999c-51e855128c5d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:57.980285 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0038ab3-111d-460c-87f3-f00b05fc20eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:57.981225 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0038ab3-111d-460c-87f3-f00b05fc20eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:57.981681 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0038ab3-111d-460c-87f3-f00b05fc20eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:57.981967 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0038ab3-111d-460c-87f3-f00b05fc20eb admin admin] PATH_INFO: `/v3/users/38fcd26af24f49cab7c231db4245b770` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:57.983295 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0038ab3-111d-460c-87f3-f00b05fc20eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=38fcd26af24f49cab7c231db4245b770)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:57.985238 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0038ab3-111d-460c-87f3-f00b05fc20eb admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:58.025059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df31c1a-5b6c-496e-b1b2-17fc0b7ed3e2 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:58.030547 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad16b1-5e15-42a8-8a99-24c27f0e298f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:58.031243 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad16b1-5e15-42a8-8a99-24c27f0e298f admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:58.031549 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad16b1-5e15-42a8-8a99-24c27f0e298f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:58.031809 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad16b1-5e15-42a8-8a99-24c27f0e298f admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:58.032539 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad16b1-5e15-42a8-8a99-24c27f0e298f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:58.034791 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad16b1-5e15-42a8-8a99-24c27f0e298f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:58.060844 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 945/1931] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:32:57 2022] POST /identity/v3/users => generated 288 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:58.061996 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad16b1-5e15-42a8-8a99-24c27f0e298f admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:58.064802 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a0038ab3-111d-460c-87f3-f00b05fc20eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38fcd26af24f49cab7c231db4245b770'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:58.083413 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-20ad16b1-5e15-42a8-8a99-24c27f0e298f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:58.086739 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a0038ab3-111d-460c-87f3-f00b05fc20eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38fcd26af24f49cab7c231db4245b770'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:58.088449 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a0038ab3-111d-460c-87f3-f00b05fc20eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38fcd26af24f49cab7c231db4245b770'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:58.089105 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a0038ab3-111d-460c-87f3-f00b05fc20eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38fcd26af24f49cab7c231db4245b770'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:58.100815 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a0038ab3-111d-460c-87f3-f00b05fc20eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38fcd26af24f49cab7c231db4245b770'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:58.106928 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad16b1-5e15-42a8-8a99-24c27f0e298f admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:58.124547 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9023bb75-7702-4114-affe-3a2065fa3e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:58.125587 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023bb75-7702-4114-affe-3a2065fa3e8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:58.127215 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023bb75-7702-4114-affe-3a2065fa3e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:58.127930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023bb75-7702-4114-affe-3a2065fa3e8a admin admin] PATH_INFO: `/v3/projects/108ee4206ce1487cbd96c16795db8b02` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:58.131153 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9023bb75-7702-4114-affe-3a2065fa3e8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=108ee4206ce1487cbd96c16795db8b02)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:58.134591 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 987/1932] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:32:57 2022] DELETE /identity/v3/users/38fcd26af24f49cab7c231db4245b770 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:58.135020 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9023bb75-7702-4114-affe-3a2065fa3e8a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:58.135755 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0038ab3-111d-460c-87f3-f00b05fc20eb admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:58.137273 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a0038ab3-111d-460c-87f3-f00b05fc20eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:58.228094 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9023bb75-7702-4114-affe-3a2065fa3e8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '108ee4206ce1487cbd96c16795db8b02', 'request_id': 'req-9023bb75-7702-4114-affe-3a2065fa3e8a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:58.234152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0038ab3-111d-460c-87f3-f00b05fc20eb admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:58.239682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9023bb75-7702-4114-affe-3a2065fa3e8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '108ee4206ce1487cbd96c16795db8b02', 'request_id': 'req-9023bb75-7702-4114-affe-3a2065fa3e8a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:58.251626 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:58.253609 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9023bb75-7702-4114-affe-3a2065fa3e8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '108ee4206ce1487cbd96c16795db8b02', 'request_id': 'req-9023bb75-7702-4114-affe-3a2065fa3e8a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:58.267494 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9023bb75-7702-4114-affe-3a2065fa3e8a admin admin] The token cache is being invalidate because project 108ee4206ce1487cbd96c16795db8b02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:32:58.267887 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9023bb75-7702-4114-affe-3a2065fa3e8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:58.268872 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9023bb75-7702-4114-affe-3a2065fa3e8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:58.271344 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 946/1933] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:32:58 2022] DELETE /identity/v3/projects/108ee4206ce1487cbd96c16795db8b02 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:58.272627 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a870f-0032-4d61-8deb-0374cd5798ff None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:58.272925 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a870f-0032-4d61-8deb-0374cd5798ff None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:58.273205 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a870f-0032-4d61-8deb-0374cd5798ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:58.294911 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-a47a870f-0032-4d61-8deb-0374cd5798ff None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:58.296298 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a47a870f-0032-4d61-8deb-0374cd5798ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:58.333788 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:DELETE=12,keystone:SELECT=83,keystone:INSERT=5 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:58.350229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3407d6-8457-4314-9cf9-7b134894a819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:58.351114 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3407d6-8457-4314-9cf9-7b134894a819 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:58.351554 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3407d6-8457-4314-9cf9-7b134894a819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:58.352034 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3407d6-8457-4314-9cf9-7b134894a819 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:58.352947 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3407d6-8457-4314-9cf9-7b134894a819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:58.354821 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3407d6-8457-4314-9cf9-7b134894a819 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:58.368595 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 988/1934] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:58 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:58.370654 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78442857-6980-4a3e-aed1-99a63ceaf62d None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:58.371043 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78442857-6980-4a3e-aed1-99a63ceaf62d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:58.371555 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78442857-6980-4a3e-aed1-99a63ceaf62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:58.395824 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-78442857-6980-4a3e-aed1-99a63ceaf62d None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:58.397118 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-78442857-6980-4a3e-aed1-99a63ceaf62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:58.405748 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 989/1935] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:58.406975 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78442857-6980-4a3e-aed1-99a63ceaf62d None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:58.416526 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-78442857-6980-4a3e-aed1-99a63ceaf62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:58.429668 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78442857-6980-4a3e-aed1-99a63ceaf62d None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:58.435574 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea279b-e547-4b9d-b770-df27aa02f3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:58.435574 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea279b-e547-4b9d-b770-df27aa02f3c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:58.435907 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea279b-e547-4b9d-b770-df27aa02f3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:58.436033 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea279b-e547-4b9d-b770-df27aa02f3c6 admin admin] PATH_INFO: `/v3/projects/6cea1b44ce3b4c8dad7f806a0b44047e` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:58.437718 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ea279b-e547-4b9d-b770-df27aa02f3c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cea1b44ce3b4c8dad7f806a0b44047e)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:58.439070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 947/1936] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:58.439726 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ea279b-e547-4b9d-b770-df27aa02f3c6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:58.440555 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a870f-0032-4d61-8deb-0374cd5798ff None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:58.442055 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a47a870f-0032-4d61-8deb-0374cd5798ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:58.468274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a870f-0032-4d61-8deb-0374cd5798ff None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:58.481449 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f929611-feee-4daf-8be5-cb87d55e25bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:58.483575 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f929611-feee-4daf-8be5-cb87d55e25bc admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:58.483888 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f929611-feee-4daf-8be5-cb87d55e25bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:58.484274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f929611-feee-4daf-8be5-cb87d55e25bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:58.486680 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f929611-feee-4daf-8be5-cb87d55e25bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:58.491150 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f929611-feee-4daf-8be5-cb87d55e25bc admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:58.516441 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 948/1937] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:58 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:58.523841 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05181f7-32cc-450f-bcc0-87c7a69e445b None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:58.524137 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05181f7-32cc-450f-bcc0-87c7a69e445b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:58.524422 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05181f7-32cc-450f-bcc0-87c7a69e445b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:58.530093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f6ea279b-e547-4b9d-b770-df27aa02f3c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cea1b44ce3b4c8dad7f806a0b44047e', 'request_id': 'req-f6ea279b-e547-4b9d-b770-df27aa02f3c6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:58.539211 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f6ea279b-e547-4b9d-b770-df27aa02f3c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cea1b44ce3b4c8dad7f806a0b44047e', 'request_id': 'req-f6ea279b-e547-4b9d-b770-df27aa02f3c6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:58.547594 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f6ea279b-e547-4b9d-b770-df27aa02f3c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cea1b44ce3b4c8dad7f806a0b44047e', 'request_id': 'req-f6ea279b-e547-4b9d-b770-df27aa02f3c6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:58.559948 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-c05181f7-32cc-450f-bcc0-87c7a69e445b None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:58.562336 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c05181f7-32cc-450f-bcc0-87c7a69e445b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:58.564344 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f6ea279b-e547-4b9d-b770-df27aa02f3c6 admin admin] The token cache is being invalidate because project 6cea1b44ce3b4c8dad7f806a0b44047e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:32:58.564727 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f6ea279b-e547-4b9d-b770-df27aa02f3c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:58.565939 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f6ea279b-e547-4b9d-b770-df27aa02f3c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:58.573039 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 990/1938] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:32:58 2022] DELETE /identity/v3/projects/6cea1b44ce3b4c8dad7f806a0b44047e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:58.573039 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea279b-e547-4b9d-b770-df27aa02f3c6 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:58.573039 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f6ea279b-e547-4b9d-b770-df27aa02f3c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:58.711662 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 949/1939] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:58.712826 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05181f7-32cc-450f-bcc0-87c7a69e445b None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:58.714471 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c05181f7-32cc-450f-bcc0-87c7a69e445b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:58.723246 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea279b-e547-4b9d-b770-df27aa02f3c6 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:58.730890 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05181f7-32cc-450f-bcc0-87c7a69e445b None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:58.731487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f609e06-9eef-4ec9-8a1c-9568fc5a1413 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:58.733256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f609e06-9eef-4ec9-8a1c-9568fc5a1413 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:58.733851 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f609e06-9eef-4ec9-8a1c-9568fc5a1413 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:58.734505 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f609e06-9eef-4ec9-8a1c-9568fc5a1413 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:58.738490 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd32e8-28d2-49e9-ac6a-f025e9f79486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:58.738803 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd32e8-28d2-49e9-ac6a-f025e9f79486 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:58.739206 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd32e8-28d2-49e9-ac6a-f025e9f79486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:58.739568 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd32e8-28d2-49e9-ac6a-f025e9f79486 admin admin] PATH_INFO: `/v3/projects/e19db6c591fe483d8597643a3f695cf6/users/5fef7e98cfca4879ab87444f4e1f69ff/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:58.740318 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6f609e06-9eef-4ec9-8a1c-9568fc5a1413 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:58.758780 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cd32e8-28d2-49e9-ac6a-f025e9f79486 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e19db6c591fe483d8597643a3f695cf6, user_id=5fef7e98cfca4879ab87444f4e1f69ff, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:58.761870 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cd32e8-28d2-49e9-ac6a-f025e9f79486 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:58.765758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f609e06-9eef-4ec9-8a1c-9568fc5a1413 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:58.768998 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f609e06-9eef-4ec9-8a1c-9568fc5a1413 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:58.780810 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 950/1940] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:58 2022] PUT /identity/v3/projects/e19db6c591fe483d8597643a3f695cf6/users/5fef7e98cfca4879ab87444f4e1f69ff/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:58.782205 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8467f911-19a6-486a-bab0-07b26ec6f2ac None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:58.782630 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8467f911-19a6-486a-bab0-07b26ec6f2ac None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:58.783047 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8467f911-19a6-486a-bab0-07b26ec6f2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:58.791368 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 991/1941] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:32:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:32:58.794577 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f609e06-9eef-4ec9-8a1c-9568fc5a1413 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:58.809020 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f609e06-9eef-4ec9-8a1c-9568fc5a1413 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:58.819065 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-8467f911-19a6-486a-bab0-07b26ec6f2ac None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:58.821041 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8467f911-19a6-486a-bab0-07b26ec6f2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:58.858805 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:58.889269 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b6c2f-52ce-4355-95f3-a029c1724c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:58.890058 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b6c2f-52ce-4355-95f3-a029c1724c83 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:58.890436 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b6c2f-52ce-4355-95f3-a029c1724c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:58.890789 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b6c2f-52ce-4355-95f3-a029c1724c83 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:58.891671 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b6c2f-52ce-4355-95f3-a029c1724c83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:32:58.892090 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b6c2f-52ce-4355-95f3-a029c1724c83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:58.893843 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b6c2f-52ce-4355-95f3-a029c1724c83 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:58.906788 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 992/1942] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:32:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:32:58.908067 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8de40-6b45-4809-82ce-9cb274934ea5 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:58.908545 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8de40-6b45-4809-82ce-9cb274934ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:58.909417 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8de40-6b45-4809-82ce-9cb274934ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:58.916888 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 951/1943] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:58.917397 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8467f911-19a6-486a-bab0-07b26ec6f2ac None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:58.928077 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8467f911-19a6-486a-bab0-07b26ec6f2ac None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:58.933763 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1791490-69ea-47b8-a5b7-c9c8252fcc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:58.934888 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1791490-69ea-47b8-a5b7-c9c8252fcc96 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:58.935216 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1791490-69ea-47b8-a5b7-c9c8252fcc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:58.935548 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1791490-69ea-47b8-a5b7-c9c8252fcc96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:58.936463 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1791490-69ea-47b8-a5b7-c9c8252fcc96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:58.938506 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1791490-69ea-47b8-a5b7-c9c8252fcc96 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:58.944402 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-05b8de40-6b45-4809-82ce-9cb274934ea5 None None] MFA Rules not processed for user `5fef7e98cfca4879ab87444f4e1f69ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:58.945746 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-05b8de40-6b45-4809-82ce-9cb274934ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:58.974805 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 952/1944] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:32:58 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:58.976054 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5353e-ae12-45cf-92bb-58efa5715e3e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:58.976350 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5353e-ae12-45cf-92bb-58efa5715e3e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:58.976622 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5353e-ae12-45cf-92bb-58efa5715e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:59.002415 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-70b5353e-ae12-45cf-92bb-58efa5715e3e None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:32:59.008192 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-70b5353e-ae12-45cf-92bb-58efa5715e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:59.031207 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:59.050816 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 993/1945] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:59.051949 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b8de40-6b45-4809-82ce-9cb274934ea5 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:59.066684 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b8de40-6b45-4809-82ce-9cb274934ea5 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:59.114406 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 953/1946] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:32:59.114406 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b5353e-ae12-45cf-92bb-58efa5715e3e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:59.127567 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b5353e-ae12-45cf-92bb-58efa5715e3e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:59.138519 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbe06b9-e6fe-4953-8f1c-e4c039666b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:59.139215 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe06b9-e6fe-4953-8f1c-e4c039666b11 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:59.139499 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe06b9-e6fe-4953-8f1c-e4c039666b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:59.139796 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe06b9-e6fe-4953-8f1c-e4c039666b11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:59.140616 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbe06b9-e6fe-4953-8f1c-e4c039666b11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:59.142671 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbe06b9-e6fe-4953-8f1c-e4c039666b11 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:59.142785 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020329d3-d044-4145-9c55-39363b0f4ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:59.143942 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020329d3-d044-4145-9c55-39363b0f4ed7 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:59.144333 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020329d3-d044-4145-9c55-39363b0f4ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:59.144710 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020329d3-d044-4145-9c55-39363b0f4ed7 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:59.145529 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020329d3-d044-4145-9c55-39363b0f4ed7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:59.147629 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020329d3-d044-4145-9c55-39363b0f4ed7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:59.171095 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 994/1947] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:32:59 2022] POST /identity/v3/users => generated 288 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:59.172126 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020329d3-d044-4145-9c55-39363b0f4ed7 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:59.178418 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 954/1948] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:32:59 2022] POST /identity/v3/projects => generated 373 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:59.180641 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbe06b9-e6fe-4953-8f1c-e4c039666b11 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:59.185309 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020329d3-d044-4145-9c55-39363b0f4ed7 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:59.253351 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:59.295200 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c01cb7-398c-4af6-802d-f1f46e522003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:59.296069 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c01cb7-398c-4af6-802d-f1f46e522003 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:59.296588 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c01cb7-398c-4af6-802d-f1f46e522003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:59.296856 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c01cb7-398c-4af6-802d-f1f46e522003 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:59.298385 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c01cb7-398c-4af6-802d-f1f46e522003 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:32:59.298957 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c01cb7-398c-4af6-802d-f1f46e522003 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:59.302396 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbe06b9-e6fe-4953-8f1c-e4c039666b11 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:59.303837 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c01cb7-398c-4af6-802d-f1f46e522003 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:59.314344 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5294df2-fb01-4223-b066-e646cf332cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:59.315174 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5294df2-fb01-4223-b066-e646cf332cb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:59.315377 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5294df2-fb01-4223-b066-e646cf332cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:59.315653 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5294df2-fb01-4223-b066-e646cf332cb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:59.317706 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5294df2-fb01-4223-b066-e646cf332cb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:59.320661 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5294df2-fb01-4223-b066-e646cf332cb5 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:59.341679 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 955/1949] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:59 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:59.343136 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5294df2-fb01-4223-b066-e646cf332cb5 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:59.349383 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 995/1950] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:32:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:32:59.350458 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c01cb7-398c-4af6-802d-f1f46e522003 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:59.363862 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5294df2-fb01-4223-b066-e646cf332cb5 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:59.369438 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4645e2f-5e39-464f-a2a4-8861a814315c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:59.370553 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4645e2f-5e39-464f-a2a4-8861a814315c admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:59.370697 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4645e2f-5e39-464f-a2a4-8861a814315c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:59.371087 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4645e2f-5e39-464f-a2a4-8861a814315c admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:59.372098 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4645e2f-5e39-464f-a2a4-8861a814315c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:59.374515 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4645e2f-5e39-464f-a2a4-8861a814315c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:59.375024 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c01cb7-398c-4af6-802d-f1f46e522003 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:59.389541 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249601c2-fc87-4ac3-9b4d-1bcf20df4cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:59.390354 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249601c2-fc87-4ac3-9b4d-1bcf20df4cc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:59.390681 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249601c2-fc87-4ac3-9b4d-1bcf20df4cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:59.391237 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249601c2-fc87-4ac3-9b4d-1bcf20df4cc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:59.391865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249601c2-fc87-4ac3-9b4d-1bcf20df4cc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:59.394815 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249601c2-fc87-4ac3-9b4d-1bcf20df4cc8 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:59.406462 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 956/1951] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:32:59 2022] POST /identity/v3/users => generated 293 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:59.406462 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4645e2f-5e39-464f-a2a4-8861a814315c admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:59.418615 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4645e2f-5e39-464f-a2a4-8861a814315c admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:59.424534 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ba378-4677-4465-ab25-38ff2e547bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:59.425376 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ba378-4677-4465-ab25-38ff2e547bc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:59.425855 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ba378-4677-4465-ab25-38ff2e547bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:59.426269 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ba378-4677-4465-ab25-38ff2e547bc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:59.427531 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631ba378-4677-4465-ab25-38ff2e547bc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:59.427985 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 996/1952] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:32:59 2022] POST /identity/v3/projects => generated 363 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:59.431505 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631ba378-4677-4465-ab25-38ff2e547bc9 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:59.432538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249601c2-fc87-4ac3-9b4d-1bcf20df4cc8 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:59.434643 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-249601c2-fc87-4ac3-9b4d-1bcf20df4cc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:59.447504 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 957/1953] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:59.447504 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ba378-4677-4465-ab25-38ff2e547bc9 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:59.460332 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ba378-4677-4465-ab25-38ff2e547bc9 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:59.513325 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249601c2-fc87-4ac3-9b4d-1bcf20df4cc8 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:59.527033 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46577e2-e78a-463f-8cac-57353eb2dfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:59.527653 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46577e2-e78a-463f-8cac-57353eb2dfdb admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:59.527936 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46577e2-e78a-463f-8cac-57353eb2dfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:59.528211 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46577e2-e78a-463f-8cac-57353eb2dfdb admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:59.528467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abcbcd9-f1d0-437a-8ce6-dbb8208a390b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:59.529115 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46577e2-e78a-463f-8cac-57353eb2dfdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:59.529749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abcbcd9-f1d0-437a-8ce6-dbb8208a390b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:59.530230 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abcbcd9-f1d0-437a-8ce6-dbb8208a390b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:59.530540 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abcbcd9-f1d0-437a-8ce6-dbb8208a390b admin admin] PATH_INFO: `/v3/projects/0f1261c2d43145e1b55b8f5d882eabb6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:59.531491 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46577e2-e78a-463f-8cac-57353eb2dfdb admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:59.532221 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abcbcd9-f1d0-437a-8ce6-dbb8208a390b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f1261c2d43145e1b55b8f5d882eabb6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:59.533547 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:59.535873 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abcbcd9-f1d0-437a-8ce6-dbb8208a390b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:59.555847 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 958/1954] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:59 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:59.557041 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46577e2-e78a-463f-8cac-57353eb2dfdb admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:59.571305 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46577e2-e78a-463f-8cac-57353eb2dfdb admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:59.578999 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d5c3d-ea9e-4f3b-bd3d-dd174b6e86e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:59.580433 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d5c3d-ea9e-4f3b-bd3d-dd174b6e86e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:59.581986 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d5c3d-ea9e-4f3b-bd3d-dd174b6e86e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:59.582880 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d5c3d-ea9e-4f3b-bd3d-dd174b6e86e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:59.583475 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d5c3d-ea9e-4f3b-bd3d-dd174b6e86e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:59.585525 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d5c3d-ea9e-4f3b-bd3d-dd174b6e86e9 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:59.627299 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9abcbcd9-f1d0-437a-8ce6-dbb8208a390b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f1261c2d43145e1b55b8f5d882eabb6', 'request_id': 'req-9abcbcd9-f1d0-437a-8ce6-dbb8208a390b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:59.645869 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 959/1955] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:32:59 2022] POST /identity/v3/users => generated 282 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:32:59.646861 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d5c3d-ea9e-4f3b-bd3d-dd174b6e86e9 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:59.648845 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9abcbcd9-f1d0-437a-8ce6-dbb8208a390b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f1261c2d43145e1b55b8f5d882eabb6', 'request_id': 'req-9abcbcd9-f1d0-437a-8ce6-dbb8208a390b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:59.660658 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9abcbcd9-f1d0-437a-8ce6-dbb8208a390b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f1261c2d43145e1b55b8f5d882eabb6', 'request_id': 'req-9abcbcd9-f1d0-437a-8ce6-dbb8208a390b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:59.668537 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d5c3d-ea9e-4f3b-bd3d-dd174b6e86e9 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:59.686969 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9abcbcd9-f1d0-437a-8ce6-dbb8208a390b admin admin] The token cache is being invalidate because project 0f1261c2d43145e1b55b8f5d882eabb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:32:59.687761 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9abcbcd9-f1d0-437a-8ce6-dbb8208a390b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:59.689448 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9abcbcd9-f1d0-437a-8ce6-dbb8208a390b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:32:59.693879 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 997/1956] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:32:59 2022] DELETE /identity/v3/projects/0f1261c2d43145e1b55b8f5d882eabb6 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:59.696009 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abcbcd9-f1d0-437a-8ce6-dbb8208a390b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:59.698531 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9abcbcd9-f1d0-437a-8ce6-dbb8208a390b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:59.766213 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b198e-d20f-4323-9cc1-35d107f2cabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:59.767171 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b198e-d20f-4323-9cc1-35d107f2cabb admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:59.767555 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b198e-d20f-4323-9cc1-35d107f2cabb admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:59.767935 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b198e-d20f-4323-9cc1-35d107f2cabb admin admin] PATH_INFO: `/v3/projects/6be4490715a14e6ba6205157a0e28743/users/9a04801eafc546058ca61be090668788/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:59.774675 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:32:59.794309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b198e-d20f-4323-9cc1-35d107f2cabb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6be4490715a14e6ba6205157a0e28743, user_id=9a04801eafc546058ca61be090668788, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:59.795847 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abcbcd9-f1d0-437a-8ce6-dbb8208a390b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:59.798946 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b198e-d20f-4323-9cc1-35d107f2cabb admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:59.802564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf142b-ef7b-488c-a098-6bed45010572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:59.803677 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf142b-ef7b-488c-a098-6bed45010572 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:59.803786 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf142b-ef7b-488c-a098-6bed45010572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:59.804068 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf142b-ef7b-488c-a098-6bed45010572 admin admin] PATH_INFO: `/v3/projects/69392f9b92f5411eb44fb644d126e31f/users/bff9f279402d45f18c19624ffb807ce4/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:59.816718 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 960/1957] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:59 2022] PUT /identity/v3/projects/6be4490715a14e6ba6205157a0e28743/users/9a04801eafc546058ca61be090668788/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:59.817912 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b198e-d20f-4323-9cc1-35d107f2cabb admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:59.820246 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-825b198e-d20f-4323-9cc1-35d107f2cabb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:59.832288 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaf142b-ef7b-488c-a098-6bed45010572 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69392f9b92f5411eb44fb644d126e31f, user_id=bff9f279402d45f18c19624ffb807ce4, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:59.835003 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaf142b-ef7b-488c-a098-6bed45010572 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:59.858359 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 998/1958] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:32:59 2022] PUT /identity/v3/projects/69392f9b92f5411eb44fb644d126e31f/users/bff9f279402d45f18c19624ffb807ce4/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:32:59.860304 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf142b-ef7b-488c-a098-6bed45010572 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:32:59.863457 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-eaaf142b-ef7b-488c-a098-6bed45010572 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:32:59.908133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b198e-d20f-4323-9cc1-35d107f2cabb admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:59.936932 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf142b-ef7b-488c-a098-6bed45010572 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:32:59.943789 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee31dd3-717d-4017-819d-7d051dce3449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:59.944716 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee31dd3-717d-4017-819d-7d051dce3449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:59.945006 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee31dd3-717d-4017-819d-7d051dce3449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:59.945277 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee31dd3-717d-4017-819d-7d051dce3449 admin admin] PATH_INFO: `/v3/users/e5ec697a98874283b2d0d23ba45e40fc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:59.946657 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee31dd3-717d-4017-819d-7d051dce3449 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5ec697a98874283b2d0d23ba45e40fc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:59.948625 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee31dd3-717d-4017-819d-7d051dce3449 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:59.974557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b9cb93-4ae7-4a4f-ab2c-8142f3f083bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:32:59.975248 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b9cb93-4ae7-4a4f-ab2c-8142f3f083bb admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:59.975527 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b9cb93-4ae7-4a4f-ab2c-8142f3f083bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:59.975800 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b9cb93-4ae7-4a4f-ab2c-8142f3f083bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:32:59.976619 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b9cb93-4ae7-4a4f-ab2c-8142f3f083bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:32:59.978318 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b9cb93-4ae7-4a4f-ab2c-8142f3f083bb admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:32:59.989917 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 961/1959] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:32:59 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:32:59.991355 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b29c1-3aac-4397-828a-7207680d7b71 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:32:59.991644 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b29c1-3aac-4397-828a-7207680d7b71 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:32:59.991918 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b29c1-3aac-4397-828a-7207680d7b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:00.001101 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:00.014778 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0ee31dd3-717d-4017-819d-7d051dce3449 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5ec697a98874283b2d0d23ba45e40fc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:00.023501 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-858b29c1-3aac-4397-828a-7207680d7b71 None None] MFA Rules not processed for user `9a04801eafc546058ca61be090668788`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:00.024959 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-858b29c1-3aac-4397-828a-7207680d7b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:00.029132 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0ee31dd3-717d-4017-819d-7d051dce3449 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5ec697a98874283b2d0d23ba45e40fc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:00.029913 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0ee31dd3-717d-4017-819d-7d051dce3449 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5ec697a98874283b2d0d23ba45e40fc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:00.030573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0ee31dd3-717d-4017-819d-7d051dce3449 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5ec697a98874283b2d0d23ba45e40fc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:00.041706 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0ee31dd3-717d-4017-819d-7d051dce3449 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5ec697a98874283b2d0d23ba45e40fc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:00.060790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 999/1960] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:32:59 2022] DELETE /identity/v3/users/e5ec697a98874283b2d0d23ba45e40fc => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:00.061700 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee31dd3-717d-4017-819d-7d051dce3449 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:00.063441 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0ee31dd3-717d-4017-819d-7d051dce3449 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:00.108578 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 962/1961] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:32:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:00.109540 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b29c1-3aac-4397-828a-7207680d7b71 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:00.110918 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-858b29c1-3aac-4397-828a-7207680d7b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:00.135671 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:00.155074 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b29c1-3aac-4397-828a-7207680d7b71 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:00.163949 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee31dd3-717d-4017-819d-7d051dce3449 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:00.167205 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36ed34-c66b-4e7a-abdd-f4bb7b1629b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:00.167916 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36ed34-c66b-4e7a-abdd-f4bb7b1629b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:00.168361 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36ed34-c66b-4e7a-abdd-f4bb7b1629b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:00.168492 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36ed34-c66b-4e7a-abdd-f4bb7b1629b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:00.169188 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b36ed34-c66b-4e7a-abdd-f4bb7b1629b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:00.171145 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b36ed34-c66b-4e7a-abdd-f4bb7b1629b7 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:00.173980 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36312df6-e45a-4a60-aebd-bd21a906b9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:00.175138 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36312df6-e45a-4a60-aebd-bd21a906b9d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:00.175525 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36312df6-e45a-4a60-aebd-bd21a906b9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:00.175901 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36312df6-e45a-4a60-aebd-bd21a906b9d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:00.176923 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36312df6-e45a-4a60-aebd-bd21a906b9d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:00.179046 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36312df6-e45a-4a60-aebd-bd21a906b9d3 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:00.189732 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 963/1962] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:00 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:33:00.191290 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36ed34-c66b-4e7a-abdd-f4bb7b1629b7 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:00.194801 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3b36ed34-c66b-4e7a-abdd-f4bb7b1629b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:00.196231 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1000/1963] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:00 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:33:00.197251 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36312df6-e45a-4a60-aebd-bd21a906b9d3 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:00.209987 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36312df6-e45a-4a60-aebd-bd21a906b9d3 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:00.215959 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26178466-bb22-470b-a8e7-e9b8b9cced74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:00.217191 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26178466-bb22-470b-a8e7-e9b8b9cced74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:00.217605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26178466-bb22-470b-a8e7-e9b8b9cced74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:00.218002 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26178466-bb22-470b-a8e7-e9b8b9cced74 admin admin] PATH_INFO: `/v3/projects/1edf1975ec2c420ca37e958daf07b7f5/users/9e382260bfd948e98320d017802ba8f3/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:00.247392 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26178466-bb22-470b-a8e7-e9b8b9cced74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1edf1975ec2c420ca37e958daf07b7f5, user_id=9e382260bfd948e98320d017802ba8f3, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:00.250670 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26178466-bb22-470b-a8e7-e9b8b9cced74 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:00.271000 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1001/1964] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:33:00 2022] PUT /identity/v3/projects/1edf1975ec2c420ca37e958daf07b7f5/users/9e382260bfd948e98320d017802ba8f3/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:00.272097 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26178466-bb22-470b-a8e7-e9b8b9cced74 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:00.317136 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36ed34-c66b-4e7a-abdd-f4bb7b1629b7 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:00.407336 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb9e471-5e2d-4061-9d02-fc45ae5f19d5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea874ca4088d4e6c83c9d0497323cf6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:00.408907 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9e471-5e2d-4061-9d02-fc45ae5f19d5 service glance] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:00.409673 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9e471-5e2d-4061-9d02-fc45ae5f19d5 service glance] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:00.410156 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9e471-5e2d-4061-9d02-fc45ae5f19d5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:00.450534 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26178466-bb22-470b-a8e7-e9b8b9cced74 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:00.493194 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0e607e-863b-4934-a283-49fe4b55d907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:00.494126 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e607e-863b-4934-a283-49fe4b55d907 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:00.494461 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e607e-863b-4934-a283-49fe4b55d907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:00.494727 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e607e-863b-4934-a283-49fe4b55d907 admin admin] PATH_INFO: `/v3/projects/69392f9b92f5411eb44fb644d126e31f/users/bff9f279402d45f18c19624ffb807ce4/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:00.496845 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0e607e-863b-4934-a283-49fe4b55d907 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69392f9b92f5411eb44fb644d126e31f, user_id=bff9f279402d45f18c19624ffb807ce4, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:00.501527 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0e607e-863b-4934-a283-49fe4b55d907 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:00.513435 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:00.517428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1002/1965] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:33:00 2022] PUT /identity/v3/projects/69392f9b92f5411eb44fb644d126e31f/users/bff9f279402d45f18c19624ffb807ce4/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:00.518830 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05810b-137d-440a-bb64-a5095d3baa5d None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:00.519262 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05810b-137d-440a-bb64-a5095d3baa5d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:00.519402 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05810b-137d-440a-bb64-a5095d3baa5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:00.535442 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb9e471-5e2d-4061-9d02-fc45ae5f19d5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:00.541535 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb9e471-5e2d-4061-9d02-fc45ae5f19d5 service glance] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:00.568673 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 964/1966] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:33:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:33:00.573170 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426dde2-1acd-4ff6-a29a-51d40c1e940a None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:00.577718 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426dde2-1acd-4ff6-a29a-51d40c1e940a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:00.578802 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426dde2-1acd-4ff6-a29a-51d40c1e940a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:00.603163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-be05810b-137d-440a-bb64-a5095d3baa5d None None] MFA Rules not processed for user `9e382260bfd948e98320d017802ba8f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:00.607071 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-be05810b-137d-440a-bb64-a5095d3baa5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:00.652821 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-3426dde2-1acd-4ff6-a29a-51d40c1e940a None None] MFA Rules not processed for user `bff9f279402d45f18c19624ffb807ce4`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:00.656058 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3426dde2-1acd-4ff6-a29a-51d40c1e940a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:00.827685 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1003/1967] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:00 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:00.840927 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be05810b-137d-440a-bb64-a5095d3baa5d None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:00.909430 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 965/1968] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:00 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:00.925256 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3426dde2-1acd-4ff6-a29a-51d40c1e940a None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:00.936486 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3426dde2-1acd-4ff6-a29a-51d40c1e940a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:00.976620 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:01.028248 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be05810b-137d-440a-bb64-a5095d3baa5d None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:01.041177 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd124f52-8bc4-4a58-b0bd-b6ae6ff658ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:01.041667 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd124f52-8bc4-4a58-b0bd-b6ae6ff658ed admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:01.041956 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd124f52-8bc4-4a58-b0bd-b6ae6ff658ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:01.042257 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd124f52-8bc4-4a58-b0bd-b6ae6ff658ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:01.042986 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd124f52-8bc4-4a58-b0bd-b6ae6ff658ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:01.044960 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd124f52-8bc4-4a58-b0bd-b6ae6ff658ed admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:01.098062 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1004/1969] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:33:00 2022] POST /identity/v3/projects => generated 363 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:33:01.098062 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd124f52-8bc4-4a58-b0bd-b6ae6ff658ed admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:01.099057 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fd124f52-8bc4-4a58-b0bd-b6ae6ff658ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:01.106893 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3426dde2-1acd-4ff6-a29a-51d40c1e940a None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:01.223076 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd124f52-8bc4-4a58-b0bd-b6ae6ff658ed admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:01.234633 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ddc206-9a73-4430-b359-ee699871c1eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:01.236379 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ddc206-9a73-4430-b359-ee699871c1eb service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:01.237194 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ddc206-9a73-4430-b359-ee699871c1eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:01.237631 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ddc206-9a73-4430-b359-ee699871c1eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:01.241042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-33ddc206-9a73-4430-b359-ee699871c1eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:01.256380 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e50244-6236-469f-9c52-318f1e29ff45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:01.257921 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e50244-6236-469f-9c52-318f1e29ff45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:01.258208 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e50244-6236-469f-9c52-318f1e29ff45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:01.258515 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e50244-6236-469f-9c52-318f1e29ff45 admin admin] PATH_INFO: `/v3/projects/154e8a03d11b47c7b76b1c410d48bc04` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:01.262009 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e50244-6236-469f-9c52-318f1e29ff45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=154e8a03d11b47c7b76b1c410d48bc04)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:01.266657 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e50244-6236-469f-9c52-318f1e29ff45 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:01.318843 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:01.339408 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ddc206-9a73-4430-b359-ee699871c1eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:01.341035 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ddc206-9a73-4430-b359-ee699871c1eb service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:01.345617 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-23e50244-6236-469f-9c52-318f1e29ff45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '154e8a03d11b47c7b76b1c410d48bc04', 'request_id': 'req-23e50244-6236-469f-9c52-318f1e29ff45'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:01.351934 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1005/1970] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:33:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:33:01.353010 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ddc206-9a73-4430-b359-ee699871c1eb service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:01.365281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-23e50244-6236-469f-9c52-318f1e29ff45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '154e8a03d11b47c7b76b1c410d48bc04', 'request_id': 'req-23e50244-6236-469f-9c52-318f1e29ff45'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:01.376730 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ddc206-9a73-4430-b359-ee699871c1eb service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:01.377968 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-23e50244-6236-469f-9c52-318f1e29ff45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '154e8a03d11b47c7b76b1c410d48bc04', 'request_id': 'req-23e50244-6236-469f-9c52-318f1e29ff45'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:01.394645 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-23e50244-6236-469f-9c52-318f1e29ff45 admin admin] The token cache is being invalidate because project 154e8a03d11b47c7b76b1c410d48bc04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:33:01.394939 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-23e50244-6236-469f-9c52-318f1e29ff45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:01.395670 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-23e50244-6236-469f-9c52-318f1e29ff45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:01.398515 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 966/1971] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:33:00 2022] DELETE /identity/v3/projects/154e8a03d11b47c7b76b1c410d48bc04 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:01.407573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e50244-6236-469f-9c52-318f1e29ff45 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:01.411089 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-23e50244-6236-469f-9c52-318f1e29ff45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:01.473920 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44f43a-a0c0-42cb-972e-85e6bfad75ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:01.474765 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44f43a-a0c0-42cb-972e-85e6bfad75ca admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:01.475489 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44f43a-a0c0-42cb-972e-85e6bfad75ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:01.476741 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44f43a-a0c0-42cb-972e-85e6bfad75ca admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:01.477620 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e44f43a-a0c0-42cb-972e-85e6bfad75ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:01.480173 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e44f43a-a0c0-42cb-972e-85e6bfad75ca admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:01.528388 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1006/1972] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:33:01 2022] POST /identity/v3/users => generated 288 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:33:01.532537 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44f43a-a0c0-42cb-972e-85e6bfad75ca admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:01.540274 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3e44f43a-a0c0-42cb-972e-85e6bfad75ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:01.545091 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e50244-6236-469f-9c52-318f1e29ff45 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:01.557917 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44f43a-a0c0-42cb-972e-85e6bfad75ca admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:01.559282 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba707fe3-5303-4b70-b774-e1a7c66625db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:01.560220 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba707fe3-5303-4b70-b774-e1a7c66625db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:01.560655 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba707fe3-5303-4b70-b774-e1a7c66625db admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:01.560943 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba707fe3-5303-4b70-b774-e1a7c66625db admin admin] PATH_INFO: `/v3/users/c366065f7bc148eba34fd9bf4c6d19d1` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:01.563404 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba707fe3-5303-4b70-b774-e1a7c66625db admin admin] RBAC: Authorizing `identity:delete_user(user_id=c366065f7bc148eba34fd9bf4c6d19d1)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:01.564659 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d04182-a2f2-493e-aedd-02da2ef6ada5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:01.565478 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba707fe3-5303-4b70-b774-e1a7c66625db admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:01.565812 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d04182-a2f2-493e-aedd-02da2ef6ada5 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:01.566490 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d04182-a2f2-493e-aedd-02da2ef6ada5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:01.566780 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d04182-a2f2-493e-aedd-02da2ef6ada5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:01.567561 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d04182-a2f2-493e-aedd-02da2ef6ada5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:01.571157 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d04182-a2f2-493e-aedd-02da2ef6ada5 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:01.583746 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1007/1973] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:01 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:33:01.593419 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d04182-a2f2-493e-aedd-02da2ef6ada5 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:01.599421 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:01.609062 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d04182-a2f2-493e-aedd-02da2ef6ada5 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:01.615429 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078712de-e0dd-447b-8118-3ad113d1597e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:01.616356 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078712de-e0dd-447b-8118-3ad113d1597e admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:01.616635 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078712de-e0dd-447b-8118-3ad113d1597e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:01.616917 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078712de-e0dd-447b-8118-3ad113d1597e admin admin] PATH_INFO: `/v3/projects/f09ba76d84e944738558e3ca43447650/users/05df40ecdd624a669b413d5081dc0cd1/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:01.638190 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ba707fe3-5303-4b70-b774-e1a7c66625db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c366065f7bc148eba34fd9bf4c6d19d1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:01.639174 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ba707fe3-5303-4b70-b774-e1a7c66625db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c366065f7bc148eba34fd9bf4c6d19d1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:01.639436 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078712de-e0dd-447b-8118-3ad113d1597e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f09ba76d84e944738558e3ca43447650, user_id=05df40ecdd624a669b413d5081dc0cd1, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:01.643093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078712de-e0dd-447b-8118-3ad113d1597e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:01.656776 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ba707fe3-5303-4b70-b774-e1a7c66625db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c366065f7bc148eba34fd9bf4c6d19d1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:01.659273 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1008/1974] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:33:01 2022] PUT /identity/v3/projects/f09ba76d84e944738558e3ca43447650/users/05df40ecdd624a669b413d5081dc0cd1/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:33:01.663594 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078712de-e0dd-447b-8118-3ad113d1597e admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:01.669843 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ba707fe3-5303-4b70-b774-e1a7c66625db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c366065f7bc148eba34fd9bf4c6d19d1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:01.679455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ba707fe3-5303-4b70-b774-e1a7c66625db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c366065f7bc148eba34fd9bf4c6d19d1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:01.688254 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 967/1975] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:33:01 2022] DELETE /identity/v3/users/c366065f7bc148eba34fd9bf4c6d19d1 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:01.752835 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078712de-e0dd-447b-8118-3ad113d1597e admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:01.881704 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead4ebd6-8e14-4695-add0-df27825c04f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:01.882327 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead4ebd6-8e14-4695-add0-df27825c04f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:01.884485 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead4ebd6-8e14-4695-add0-df27825c04f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:01.884952 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead4ebd6-8e14-4695-add0-df27825c04f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:01.886090 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead4ebd6-8e14-4695-add0-df27825c04f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:01.900615 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead4ebd6-8e14-4695-add0-df27825c04f6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:01.914937 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1009/1976] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:01 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:33:01.919995 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba707fe3-5303-4b70-b774-e1a7c66625db admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:01.923441 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ba707fe3-5303-4b70-b774-e1a7c66625db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:01.942404 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba707fe3-5303-4b70-b774-e1a7c66625db admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:01.949958 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e8d86-3c2b-446b-bfd0-71d546cc9c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:01.950999 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e8d86-3c2b-446b-bfd0-71d546cc9c2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:01.951350 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e8d86-3c2b-446b-bfd0-71d546cc9c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:01.951821 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e8d86-3c2b-446b-bfd0-71d546cc9c2f admin admin] PATH_INFO: `/v3/projects/f09ba76d84e944738558e3ca43447650/users/05df40ecdd624a669b413d5081dc0cd1/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:01.958747 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48e8d86-3c2b-446b-bfd0-71d546cc9c2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f09ba76d84e944738558e3ca43447650, user_id=05df40ecdd624a669b413d5081dc0cd1, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:01.965744 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48e8d86-3c2b-446b-bfd0-71d546cc9c2f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:01.981053 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 968/1977] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:33:01 2022] PUT /identity/v3/projects/f09ba76d84e944738558e3ca43447650/users/05df40ecdd624a669b413d5081dc0cd1/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:01.991795 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25678e71-c1c9-4341-abd1-7f2f1623ee1e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:01.992539 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25678e71-c1c9-4341-abd1-7f2f1623ee1e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:01.992836 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25678e71-c1c9-4341-abd1-7f2f1623ee1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:02.025045 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-25678e71-c1c9-4341-abd1-7f2f1623ee1e None None] MFA Rules not processed for user `05df40ecdd624a669b413d5081dc0cd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:02.026322 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-25678e71-c1c9-4341-abd1-7f2f1623ee1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:02.110013 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1010/1978] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:01 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:02.124644 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e8d86-3c2b-446b-bfd0-71d546cc9c2f admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:02.126846 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c48e8d86-3c2b-446b-bfd0-71d546cc9c2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:02.142107 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25678e71-c1c9-4341-abd1-7f2f1623ee1e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:02.143999 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-25678e71-c1c9-4341-abd1-7f2f1623ee1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:02.156201 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:02.168114 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:02.221075 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e8d86-3c2b-446b-bfd0-71d546cc9c2f admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:02.228628 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980cbd6-e08c-4c1e-81db-98bc62db44d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:02.229562 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980cbd6-e08c-4c1e-81db-98bc62db44d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:02.230106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980cbd6-e08c-4c1e-81db-98bc62db44d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:02.230558 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980cbd6-e08c-4c1e-81db-98bc62db44d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:02.233521 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25678e71-c1c9-4341-abd1-7f2f1623ee1e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:02.241161 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5473c1e3-9ec3-4c2c-8b19-bd94c1a301fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:02.242264 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473c1e3-9ec3-4c2c-8b19-bd94c1a301fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:02.242656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473c1e3-9ec3-4c2c-8b19-bd94c1a301fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:02.243004 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473c1e3-9ec3-4c2c-8b19-bd94c1a301fd admin admin] PATH_INFO: `/v3/projects/81525ca1a71b40e493bf8f4966909f2c` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:02.244415 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5473c1e3-9ec3-4c2c-8b19-bd94c1a301fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=81525ca1a71b40e493bf8f4966909f2c)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:02.246881 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5473c1e3-9ec3-4c2c-8b19-bd94c1a301fd admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:02.253726 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1980cbd6-e08c-4c1e-81db-98bc62db44d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:02.256816 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1980cbd6-e08c-4c1e-81db-98bc62db44d7 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:02.270961 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 969/1979] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:33:02 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:33:02.323261 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5473c1e3-9ec3-4c2c-8b19-bd94c1a301fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81525ca1a71b40e493bf8f4966909f2c', 'request_id': 'req-5473c1e3-9ec3-4c2c-8b19-bd94c1a301fd'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:02.332627 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5473c1e3-9ec3-4c2c-8b19-bd94c1a301fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81525ca1a71b40e493bf8f4966909f2c', 'request_id': 'req-5473c1e3-9ec3-4c2c-8b19-bd94c1a301fd'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:02.341631 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5473c1e3-9ec3-4c2c-8b19-bd94c1a301fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81525ca1a71b40e493bf8f4966909f2c', 'request_id': 'req-5473c1e3-9ec3-4c2c-8b19-bd94c1a301fd'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:02.371832 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5473c1e3-9ec3-4c2c-8b19-bd94c1a301fd admin admin] The token cache is being invalidate because project 81525ca1a71b40e493bf8f4966909f2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:33:02.372432 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5473c1e3-9ec3-4c2c-8b19-bd94c1a301fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:02.373675 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5473c1e3-9ec3-4c2c-8b19-bd94c1a301fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:02.380014 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1011/1980] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:33:02 2022] DELETE /identity/v3/projects/81525ca1a71b40e493bf8f4966909f2c => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:02.386061 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfce6c2-9b74-47e6-8d13-b8d2d5bdbb81 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:02.387564 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfce6c2-9b74-47e6-8d13-b8d2d5bdbb81 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:02.387564 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfce6c2-9b74-47e6-8d13-b8d2d5bdbb81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:02.419096 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-dbfce6c2-9b74-47e6-8d13-b8d2d5bdbb81 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:02.425053 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-dbfce6c2-9b74-47e6-8d13-b8d2d5bdbb81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:02.514480 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 970/1981] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:02.526806 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4563c-aa9b-4b25-ae57-e654728e969a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:02.527110 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4563c-aa9b-4b25-ae57-e654728e969a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:02.527393 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4563c-aa9b-4b25-ae57-e654728e969a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:02.571817 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-3bd4563c-aa9b-4b25-ae57-e654728e969a None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:02.573085 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3bd4563c-aa9b-4b25-ae57-e654728e969a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:02.582657 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1012/1982] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:02.589198 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfce6c2-9b74-47e6-8d13-b8d2d5bdbb81 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:02.603782 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfce6c2-9b74-47e6-8d13-b8d2d5bdbb81 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:02.609484 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebfa4e4-99c2-4de5-aeef-f4e0fe3269c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:02.610426 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebfa4e4-99c2-4de5-aeef-f4e0fe3269c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:02.610838 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebfa4e4-99c2-4de5-aeef-f4e0fe3269c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:02.610989 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebfa4e4-99c2-4de5-aeef-f4e0fe3269c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:02.611838 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebfa4e4-99c2-4de5-aeef-f4e0fe3269c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:33:02.612164 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebfa4e4-99c2-4de5-aeef-f4e0fe3269c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:02.613824 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebfa4e4-99c2-4de5-aeef-f4e0fe3269c1 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:02.629041 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 971/1983] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:33:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:33:02.634772 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4563c-aa9b-4b25-ae57-e654728e969a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:02.647436 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4563c-aa9b-4b25-ae57-e654728e969a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:02.653519 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f8a624-226b-4306-9be4-302dbc54f10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:02.654297 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f8a624-226b-4306-9be4-302dbc54f10f admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:02.654584 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f8a624-226b-4306-9be4-302dbc54f10f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:02.654937 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f8a624-226b-4306-9be4-302dbc54f10f admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:02.655574 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f8a624-226b-4306-9be4-302dbc54f10f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:02.657619 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f8a624-226b-4306-9be4-302dbc54f10f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:02.708382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebfa4e4-99c2-4de5-aeef-f4e0fe3269c1 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:02.709673 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1013/1984] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:33:02 2022] POST /identity/v3/projects => generated 381 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:33:02.720443 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebfa4e4-99c2-4de5-aeef-f4e0fe3269c1 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:02.789454 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573fd4a3-a97b-4e89-8b85-3184da30d8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:02.790201 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573fd4a3-a97b-4e89-8b85-3184da30d8ac admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:02.790484 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573fd4a3-a97b-4e89-8b85-3184da30d8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:02.790763 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573fd4a3-a97b-4e89-8b85-3184da30d8ac admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:02.791481 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573fd4a3-a97b-4e89-8b85-3184da30d8ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:02.793486 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573fd4a3-a97b-4e89-8b85-3184da30d8ac admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:02.818539 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 972/1985] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:33:02 2022] POST /identity/v3/users => generated 291 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:33:02.823229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f8a624-226b-4306-9be4-302dbc54f10f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:02.844281 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f8a624-226b-4306-9be4-302dbc54f10f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:02.850740 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacaf2d0-b4c1-4ebd-9207-9bbf97400457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:02.852022 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacaf2d0-b4c1-4ebd-9207-9bbf97400457 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:02.852453 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacaf2d0-b4c1-4ebd-9207-9bbf97400457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:02.852731 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacaf2d0-b4c1-4ebd-9207-9bbf97400457 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:02.853528 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacaf2d0-b4c1-4ebd-9207-9bbf97400457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:02.855190 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacaf2d0-b4c1-4ebd-9207-9bbf97400457 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:02.873886 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1014/1986] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:33:02.875187 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573fd4a3-a97b-4e89-8b85-3184da30d8ac admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:02.886540 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573fd4a3-a97b-4e89-8b85-3184da30d8ac admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:02.891343 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacaf2d0-b4c1-4ebd-9207-9bbf97400457 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:02.893335 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233e5ec-b0ed-468d-b346-6b8ad013daf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:02.893970 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-bacaf2d0-b4c1-4ebd-9207-9bbf97400457 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:02.894136 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233e5ec-b0ed-468d-b346-6b8ad013daf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:02.894875 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233e5ec-b0ed-468d-b346-6b8ad013daf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:02.894875 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233e5ec-b0ed-468d-b346-6b8ad013daf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:02.895486 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2233e5ec-b0ed-468d-b346-6b8ad013daf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:02.897297 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2233e5ec-b0ed-468d-b346-6b8ad013daf2 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:02.909741 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 973/1987] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:33:02.916236 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233e5ec-b0ed-468d-b346-6b8ad013daf2 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:02.932698 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacaf2d0-b4c1-4ebd-9207-9bbf97400457 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:02.933473 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233e5ec-b0ed-468d-b346-6b8ad013daf2 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:02.938323 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cbf4c3-e187-4f92-b4da-c6cd620be7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:02.939436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbf4c3-e187-4f92-b4da-c6cd620be7f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:02.939718 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbf4c3-e187-4f92-b4da-c6cd620be7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:02.940306 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbf4c3-e187-4f92-b4da-c6cd620be7f5 admin admin] PATH_INFO: `/v3/projects/c7f22a04dff547b989a511628fcdf292/users/c210a0109d3e468bb33f139af3411221/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:02.944740 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38faf551-0b63-48d3-befc-d1b91cd92f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:02.945447 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38faf551-0b63-48d3-befc-d1b91cd92f5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:02.946367 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38faf551-0b63-48d3-befc-d1b91cd92f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:02.947152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38faf551-0b63-48d3-befc-d1b91cd92f5f admin admin] PATH_INFO: `/v3/users/9e382260bfd948e98320d017802ba8f3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:02.950987 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38faf551-0b63-48d3-befc-d1b91cd92f5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e382260bfd948e98320d017802ba8f3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:02.954140 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38faf551-0b63-48d3-befc-d1b91cd92f5f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:02.965930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cbf4c3-e187-4f92-b4da-c6cd620be7f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7f22a04dff547b989a511628fcdf292, user_id=c210a0109d3e468bb33f139af3411221, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:02.969079 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cbf4c3-e187-4f92-b4da-c6cd620be7f5 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:02.999544 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 974/1988] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:33:02 2022] PUT /identity/v3/projects/c7f22a04dff547b989a511628fcdf292/users/c210a0109d3e468bb33f139af3411221/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:03.011583 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b5b9c-498d-40da-a506-2eae6b0458c4 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:03.011897 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b5b9c-498d-40da-a506-2eae6b0458c4 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:03.012144 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b5b9c-498d-40da-a506-2eae6b0458c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:03.031161 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:03.031360 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-38faf551-0b63-48d3-befc-d1b91cd92f5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e382260bfd948e98320d017802ba8f3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:03.038625 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:03.050432 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-9b9b5b9c-498d-40da-a506-2eae6b0458c4 None None] MFA Rules not processed for user `c210a0109d3e468bb33f139af3411221`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:03.052061 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9b9b5b9c-498d-40da-a506-2eae6b0458c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:03.057883 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-38faf551-0b63-48d3-befc-d1b91cd92f5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e382260bfd948e98320d017802ba8f3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:03.058795 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-38faf551-0b63-48d3-befc-d1b91cd92f5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e382260bfd948e98320d017802ba8f3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:03.059512 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-38faf551-0b63-48d3-befc-d1b91cd92f5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e382260bfd948e98320d017802ba8f3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:03.068245 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-38faf551-0b63-48d3-befc-d1b91cd92f5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e382260bfd948e98320d017802ba8f3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:03.091836 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1015/1989] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:33:02 2022] DELETE /identity/v3/users/9e382260bfd948e98320d017802ba8f3 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:03.096973 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72537cb-e794-4f5d-8648-73622c1166b1 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:03.097347 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72537cb-e794-4f5d-8648-73622c1166b1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:03.097695 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72537cb-e794-4f5d-8648-73622c1166b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:03.125677 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d72537cb-e794-4f5d-8648-73622c1166b1 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:03.127270 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d72537cb-e794-4f5d-8648-73622c1166b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:03.146478 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 975/1990] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:33:03.151434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9b5b9c-498d-40da-a506-2eae6b0458c4 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:03.152868 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9b9b5b9c-498d-40da-a506-2eae6b0458c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:03.215384 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1016/1991] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:03.233974 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72537cb-e794-4f5d-8648-73622c1166b1 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:03.236533 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d72537cb-e794-4f5d-8648-73622c1166b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:03.237596 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9b5b9c-498d-40da-a506-2eae6b0458c4 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:03.242203 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe6f89d-0662-4def-9c18-b324896bc883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:03.243500 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe6f89d-0662-4def-9c18-b324896bc883 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:03.243500 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe6f89d-0662-4def-9c18-b324896bc883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:03.243500 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe6f89d-0662-4def-9c18-b324896bc883 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:03.244166 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe6f89d-0662-4def-9c18-b324896bc883 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:03.246226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe6f89d-0662-4def-9c18-b324896bc883 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:03.277106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 976/1992] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:33:03 2022] POST /identity/v3/projects => generated 379 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:33:03.291493 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe6f89d-0662-4def-9c18-b324896bc883 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:03.307896 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72537cb-e794-4f5d-8648-73622c1166b1 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:03.355144 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe6f89d-0662-4def-9c18-b324896bc883 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:03.361008 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f95ec-db41-4da5-a490-69f353a4511c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:03.361747 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f95ec-db41-4da5-a490-69f353a4511c admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:03.362033 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f95ec-db41-4da5-a490-69f353a4511c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:03.362379 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f95ec-db41-4da5-a490-69f353a4511c admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:03.363088 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7f95ec-db41-4da5-a490-69f353a4511c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:03.365242 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7f95ec-db41-4da5-a490-69f353a4511c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:03.366866 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5573177d-13f7-4cbf-8ac0-a6dc841da26f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:03.367527 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5573177d-13f7-4cbf-8ac0-a6dc841da26f service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:03.367808 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5573177d-13f7-4cbf-8ac0-a6dc841da26f service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:03.368089 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5573177d-13f7-4cbf-8ac0-a6dc841da26f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:03.377772 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5573177d-13f7-4cbf-8ac0-a6dc841da26f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:03.379540 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5573177d-13f7-4cbf-8ac0-a6dc841da26f service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:03.387063 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 977/1993] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:33:03 2022] POST /identity/v3/users => generated 296 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:33:03.389898 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1017/1994] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:33:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:33:03.392041 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f95ec-db41-4da5-a490-69f353a4511c admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:03.403921 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f95ec-db41-4da5-a490-69f353a4511c admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:03.409069 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8cb2e2-a2f2-4243-8e83-3ab849481622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:03.409760 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8cb2e2-a2f2-4243-8e83-3ab849481622 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:03.410038 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8cb2e2-a2f2-4243-8e83-3ab849481622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:03.410345 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8cb2e2-a2f2-4243-8e83-3ab849481622 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:03.411111 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8cb2e2-a2f2-4243-8e83-3ab849481622 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:03.412714 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8cb2e2-a2f2-4243-8e83-3ab849481622 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:03.419050 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:03.425453 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 978/1995] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 21:33:03.429569 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5573177d-13f7-4cbf-8ac0-a6dc841da26f service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:03.440769 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5573177d-13f7-4cbf-8ac0-a6dc841da26f service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:03.446819 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f35aa9-bc2d-4b4f-944e-6c237589f0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:03.447844 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f35aa9-bc2d-4b4f-944e-6c237589f0bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:03.448264 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f35aa9-bc2d-4b4f-944e-6c237589f0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:03.448444 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f35aa9-bc2d-4b4f-944e-6c237589f0bd admin admin] PATH_INFO: `/v3/projects/942fed1f8cbc4499a16bd2199a027a0f/users/8c3933fd54294c00b09738b92db89b7a/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:03.468128 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f35aa9-bc2d-4b4f-944e-6c237589f0bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=942fed1f8cbc4499a16bd2199a027a0f, user_id=8c3933fd54294c00b09738b92db89b7a, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:03.471394 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f35aa9-bc2d-4b4f-944e-6c237589f0bd admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:03.487143 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1018/1996] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:33:03 2022] PUT /identity/v3/projects/942fed1f8cbc4499a16bd2199a027a0f/users/8c3933fd54294c00b09738b92db89b7a/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:03.491058 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8cb2e2-a2f2-4243-8e83-3ab849481622 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:03.502846 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8cb2e2-a2f2-4243-8e83-3ab849481622 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:03.572856 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56afa2b2-a0a3-4871-9167-b033e16c7867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:03.573646 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afa2b2-a0a3-4871-9167-b033e16c7867 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:03.573997 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afa2b2-a0a3-4871-9167-b033e16c7867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:03.574347 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afa2b2-a0a3-4871-9167-b033e16c7867 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:03.575214 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56afa2b2-a0a3-4871-9167-b033e16c7867 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:03.576903 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56afa2b2-a0a3-4871-9167-b033e16c7867 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:03.589921 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 979/1997] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:33:03.593714 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f35aa9-bc2d-4b4f-944e-6c237589f0bd admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:03.601484 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:03.605738 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f35aa9-bc2d-4b4f-944e-6c237589f0bd admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:03.613001 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa415f-045e-4a09-9385-c2d46741e12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:03.614157 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa415f-045e-4a09-9385-c2d46741e12d admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:03.614726 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa415f-045e-4a09-9385-c2d46741e12d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:03.615012 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa415f-045e-4a09-9385-c2d46741e12d admin admin] PATH_INFO: `/v3/projects/942fed1f8cbc4499a16bd2199a027a0f/users/8c3933fd54294c00b09738b92db89b7a/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:03.617474 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa415f-045e-4a09-9385-c2d46741e12d admin admin] RBAC: Authorizing `identity:create_grant(project_id=942fed1f8cbc4499a16bd2199a027a0f, user_id=8c3933fd54294c00b09738b92db89b7a, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:03.620183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa415f-045e-4a09-9385-c2d46741e12d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:03.633981 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1019/1998] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:33:03 2022] PUT /identity/v3/projects/942fed1f8cbc4499a16bd2199a027a0f/users/8c3933fd54294c00b09738b92db89b7a/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:33:03.648006 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df3055f-8dc3-402f-9790-a9a2b09f59fe None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:03.648297 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df3055f-8dc3-402f-9790-a9a2b09f59fe None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:03.648567 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df3055f-8dc3-402f-9790-a9a2b09f59fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:03.684757 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-1df3055f-8dc3-402f-9790-a9a2b09f59fe None None] MFA Rules not processed for user `8c3933fd54294c00b09738b92db89b7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:03.686028 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1df3055f-8dc3-402f-9790-a9a2b09f59fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:03.755707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa415f-045e-4a09-9385-c2d46741e12d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:03.757704 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-eeaa415f-045e-4a09-9385-c2d46741e12d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:03.789209 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 980/1999] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:03 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:03.801840 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df3055f-8dc3-402f-9790-a9a2b09f59fe None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:03.803941 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1df3055f-8dc3-402f-9790-a9a2b09f59fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:03.837384 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa415f-045e-4a09-9385-c2d46741e12d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:03.842860 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6831bb-24c1-45dd-909e-228752f95995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:03.843760 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6831bb-24c1-45dd-909e-228752f95995 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:03.844046 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6831bb-24c1-45dd-909e-228752f95995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:03.844323 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6831bb-24c1-45dd-909e-228752f95995 admin admin] PATH_INFO: `/v3/projects/1edf1975ec2c420ca37e958daf07b7f5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:03.845935 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6831bb-24c1-45dd-909e-228752f95995 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1edf1975ec2c420ca37e958daf07b7f5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:03.848493 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6831bb-24c1-45dd-909e-228752f95995 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:03.870762 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df3055f-8dc3-402f-9790-a9a2b09f59fe None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:03.874595 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67078297-0e87-46b1-b2ed-11abcd260e86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:03.875268 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67078297-0e87-46b1-b2ed-11abcd260e86 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:03.875549 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67078297-0e87-46b1-b2ed-11abcd260e86 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:03.875821 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67078297-0e87-46b1-b2ed-11abcd260e86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:03.886278 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67078297-0e87-46b1-b2ed-11abcd260e86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:03.887717 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67078297-0e87-46b1-b2ed-11abcd260e86 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:03.897874 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 981/2000] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:33:03 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:33:03.901802 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9e6831bb-24c1-45dd-909e-228752f95995 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1edf1975ec2c420ca37e958daf07b7f5', 'request_id': 'req-9e6831bb-24c1-45dd-909e-228752f95995'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:03.912313 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9e6831bb-24c1-45dd-909e-228752f95995 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1edf1975ec2c420ca37e958daf07b7f5', 'request_id': 'req-9e6831bb-24c1-45dd-909e-228752f95995'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:03.919554 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9e6831bb-24c1-45dd-909e-228752f95995 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1edf1975ec2c420ca37e958daf07b7f5', 'request_id': 'req-9e6831bb-24c1-45dd-909e-228752f95995'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:03.940887 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9e6831bb-24c1-45dd-909e-228752f95995 admin admin] The token cache is being invalidate because project 1edf1975ec2c420ca37e958daf07b7f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:33:03.941190 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9e6831bb-24c1-45dd-909e-228752f95995 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:03.942019 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9e6831bb-24c1-45dd-909e-228752f95995 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:03.943719 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1020/2001] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:33:03 2022] DELETE /identity/v3/projects/1edf1975ec2c420ca37e958daf07b7f5 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:33:03.948174 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67078297-0e87-46b1-b2ed-11abcd260e86 service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:03.960248 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67078297-0e87-46b1-b2ed-11abcd260e86 service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:03.991637 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:04.046796 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f6c56-fdb4-4563-a9b3-efb8e1aa1f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:04.048210 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f6c56-fdb4-4563-a9b3-efb8e1aa1f13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:04.048749 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f6c56-fdb4-4563-a9b3-efb8e1aa1f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:04.049206 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f6c56-fdb4-4563-a9b3-efb8e1aa1f13 admin admin] PATH_INFO: `/v3/users/05df40ecdd624a669b413d5081dc0cd1` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:04.050930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f6c56-fdb4-4563-a9b3-efb8e1aa1f13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05df40ecdd624a669b413d5081dc0cd1)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:04.053637 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f6c56-fdb4-4563-a9b3-efb8e1aa1f13 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:04.118976 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d85f6c56-fdb4-4563-a9b3-efb8e1aa1f13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05df40ecdd624a669b413d5081dc0cd1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:04.119778 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d85f6c56-fdb4-4563-a9b3-efb8e1aa1f13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05df40ecdd624a669b413d5081dc0cd1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:04.150313 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d85f6c56-fdb4-4563-a9b3-efb8e1aa1f13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05df40ecdd624a669b413d5081dc0cd1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:04.167747 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d85f6c56-fdb4-4563-a9b3-efb8e1aa1f13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05df40ecdd624a669b413d5081dc0cd1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:04.192199 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d85f6c56-fdb4-4563-a9b3-efb8e1aa1f13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05df40ecdd624a669b413d5081dc0cd1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:04.196807 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6831bb-24c1-45dd-909e-228752f95995 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:04.199138 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9e6831bb-24c1-45dd-909e-228752f95995 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:04.203859 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 982/2002] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:33:03 2022] DELETE /identity/v3/users/05df40ecdd624a669b413d5081dc0cd1 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:33:04.221527 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6831bb-24c1-45dd-909e-228752f95995 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:04.329154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd454d0-9bfb-47da-bb1a-b5f253f7a886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:04.332394 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd454d0-9bfb-47da-bb1a-b5f253f7a886 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:04.333573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd454d0-9bfb-47da-bb1a-b5f253f7a886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:04.333869 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd454d0-9bfb-47da-bb1a-b5f253f7a886 admin admin] PATH_INFO: `/v3/users/5fef7e98cfca4879ab87444f4e1f69ff` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:04.338102 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd454d0-9bfb-47da-bb1a-b5f253f7a886 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fef7e98cfca4879ab87444f4e1f69ff)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:04.343329 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd454d0-9bfb-47da-bb1a-b5f253f7a886 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:04.395341 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:04.439539 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-bdd454d0-9bfb-47da-bb1a-b5f253f7a886 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fef7e98cfca4879ab87444f4e1f69ff'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:04.466181 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-bdd454d0-9bfb-47da-bb1a-b5f253f7a886 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fef7e98cfca4879ab87444f4e1f69ff'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:04.470911 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-bdd454d0-9bfb-47da-bb1a-b5f253f7a886 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fef7e98cfca4879ab87444f4e1f69ff'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:04.471719 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-bdd454d0-9bfb-47da-bb1a-b5f253f7a886 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fef7e98cfca4879ab87444f4e1f69ff'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:04.483625 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-bdd454d0-9bfb-47da-bb1a-b5f253f7a886 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fef7e98cfca4879ab87444f4e1f69ff'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:04.494146 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f6c56-fdb4-4563-a9b3-efb8e1aa1f13 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:04.496452 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d85f6c56-fdb4-4563-a9b3-efb8e1aa1f13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:04.508243 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1021/2003] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:33:04 2022] DELETE /identity/v3/users/5fef7e98cfca4879ab87444f4e1f69ff => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:04.513755 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1175188d-9f0d-4c28-9295-5574219c89ad None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:04.514069 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1175188d-9f0d-4c28-9295-5574219c89ad None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:04.515370 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1175188d-9f0d-4c28-9295-5574219c89ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:04.519351 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f6c56-fdb4-4563-a9b3-efb8e1aa1f13 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:04.540226 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-1175188d-9f0d-4c28-9295-5574219c89ad None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:04.541482 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1175188d-9f0d-4c28-9295-5574219c89ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:04.625958 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1022/2004] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:33:04.628522 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bcb17c-e168-4709-887b-5602c7858621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:04.634792 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bcb17c-e168-4709-887b-5602c7858621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:04.635845 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bcb17c-e168-4709-887b-5602c7858621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:04.636387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bcb17c-e168-4709-887b-5602c7858621 admin admin] PATH_INFO: `/v3/users/c210a0109d3e468bb33f139af3411221` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:04.639199 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bcb17c-e168-4709-887b-5602c7858621 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c210a0109d3e468bb33f139af3411221)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:04.641765 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bcb17c-e168-4709-887b-5602c7858621 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:04.644988 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1175188d-9f0d-4c28-9295-5574219c89ad None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:04.646925 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1175188d-9f0d-4c28-9295-5574219c89ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:04.735500 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:04.738363 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d1bcb17c-e168-4709-887b-5602c7858621 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c210a0109d3e468bb33f139af3411221'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:04.739128 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d1bcb17c-e168-4709-887b-5602c7858621 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c210a0109d3e468bb33f139af3411221'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:04.747730 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1175188d-9f0d-4c28-9295-5574219c89ad None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:04.753111 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f3520-2fee-42a5-8a0a-24c7b0afa0e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:04.754064 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f3520-2fee-42a5-8a0a-24c7b0afa0e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:04.754487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f3520-2fee-42a5-8a0a-24c7b0afa0e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:04.754841 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f3520-2fee-42a5-8a0a-24c7b0afa0e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:04.758795 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d1bcb17c-e168-4709-887b-5602c7858621 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c210a0109d3e468bb33f139af3411221'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:04.768434 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453f3520-2fee-42a5-8a0a-24c7b0afa0e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:04.771007 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453f3520-2fee-42a5-8a0a-24c7b0afa0e0 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:04.773765 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d1bcb17c-e168-4709-887b-5602c7858621 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c210a0109d3e468bb33f139af3411221'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:04.783211 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1023/2005] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:33:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:33:04.791314 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d1bcb17c-e168-4709-887b-5602c7858621 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c210a0109d3e468bb33f139af3411221'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:04.810989 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 983/2006] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:33:04 2022] DELETE /identity/v3/users/c210a0109d3e468bb33f139af3411221 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:33:04.816048 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4cc99d-bcab-4d79-b747-0968a8a12cb3 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:04.816596 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4cc99d-bcab-4d79-b747-0968a8a12cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:04.817169 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4cc99d-bcab-4d79-b747-0968a8a12cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:04.833571 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bcb17c-e168-4709-887b-5602c7858621 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:04.835259 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d1bcb17c-e168-4709-887b-5602c7858621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:04.850262 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-7f4cc99d-bcab-4d79-b747-0968a8a12cb3 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:04.851618 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7f4cc99d-bcab-4d79-b747-0968a8a12cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:04.921295 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:04.957593 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1024/2007] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:04.980005 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4cc99d-bcab-4d79-b747-0968a8a12cb3 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:04.988824 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7f4cc99d-bcab-4d79-b747-0968a8a12cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:05.005332 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bcb17c-e168-4709-887b-5602c7858621 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:05.013883 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55f6c9-13fd-450f-a9b2-9e2b0f29b42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:05.014919 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55f6c9-13fd-450f-a9b2-9e2b0f29b42c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:05.015273 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55f6c9-13fd-450f-a9b2-9e2b0f29b42c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:05.015613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55f6c9-13fd-450f-a9b2-9e2b0f29b42c admin admin] PATH_INFO: `/v3/projects/f09ba76d84e944738558e3ca43447650` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:05.018449 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd55f6c9-13fd-450f-a9b2-9e2b0f29b42c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f09ba76d84e944738558e3ca43447650)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:05.021455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd55f6c9-13fd-450f-a9b2-9e2b0f29b42c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:05.103148 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bd55f6c9-13fd-450f-a9b2-9e2b0f29b42c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f09ba76d84e944738558e3ca43447650', 'request_id': 'req-bd55f6c9-13fd-450f-a9b2-9e2b0f29b42c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:05.108026 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4cc99d-bcab-4d79-b747-0968a8a12cb3 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:05.115169 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bd55f6c9-13fd-450f-a9b2-9e2b0f29b42c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f09ba76d84e944738558e3ca43447650', 'request_id': 'req-bd55f6c9-13fd-450f-a9b2-9e2b0f29b42c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:05.152337 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bd55f6c9-13fd-450f-a9b2-9e2b0f29b42c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f09ba76d84e944738558e3ca43447650', 'request_id': 'req-bd55f6c9-13fd-450f-a9b2-9e2b0f29b42c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:05.169171 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bd55f6c9-13fd-450f-a9b2-9e2b0f29b42c admin admin] The token cache is being invalidate because project f09ba76d84e944738558e3ca43447650 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:33:05.169524 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bd55f6c9-13fd-450f-a9b2-9e2b0f29b42c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:05.170182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bd55f6c9-13fd-450f-a9b2-9e2b0f29b42c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:05.176620 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 984/2008] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:33:04 2022] DELETE /identity/v3/projects/f09ba76d84e944738558e3ca43447650 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:33:05.179882 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b3a58-7c3d-4fbf-9a08-f6aa84dd8c6c None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:05.180172 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b3a58-7c3d-4fbf-9a08-f6aa84dd8c6c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:05.180452 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b3a58-7c3d-4fbf-9a08-f6aa84dd8c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:05.203703 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474c3c8-74e6-42db-8458-03c160193f96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:05.204344 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474c3c8-74e6-42db-8458-03c160193f96 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:05.204631 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474c3c8-74e6-42db-8458-03c160193f96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:05.204916 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474c3c8-74e6-42db-8458-03c160193f96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:05.207282 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-742b3a58-7c3d-4fbf-9a08-f6aa84dd8c6c None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:05.208561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-742b3a58-7c3d-4fbf-9a08-f6aa84dd8c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:05.212603 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c474c3c8-74e6-42db-8458-03c160193f96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:05.302614 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:05.321555 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474c3c8-74e6-42db-8458-03c160193f96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:05.323082 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474c3c8-74e6-42db-8458-03c160193f96 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:05.334073 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1025/2009] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:33:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:33:05.336316 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 985/2010] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:05.338182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b3a58-7c3d-4fbf-9a08-f6aa84dd8c6c None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:05.340385 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-742b3a58-7c3d-4fbf-9a08-f6aa84dd8c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:05.352084 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2a605-4b84-4f43-85f3-0594ad1703fc None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:05.352377 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2a605-4b84-4f43-85f3-0594ad1703fc None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:05.352663 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2a605-4b84-4f43-85f3-0594ad1703fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:05.357703 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b3a58-7c3d-4fbf-9a08-f6aa84dd8c6c None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:05.363884 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c68257-1694-4017-9e18-63e09727d6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:05.364795 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c68257-1694-4017-9e18-63e09727d6d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:05.365078 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c68257-1694-4017-9e18-63e09727d6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:05.365356 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c68257-1694-4017-9e18-63e09727d6d8 admin admin] PATH_INFO: `/v3/projects/e19db6c591fe483d8597643a3f695cf6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:05.366964 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c68257-1694-4017-9e18-63e09727d6d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e19db6c591fe483d8597643a3f695cf6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:05.369246 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c68257-1694-4017-9e18-63e09727d6d8 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:05.385001 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e5a2a605-4b84-4f43-85f3-0594ad1703fc None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:05.386639 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e5a2a605-4b84-4f43-85f3-0594ad1703fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:05.402435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1026/2011] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:05.407265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a2a605-4b84-4f43-85f3-0594ad1703fc None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:05.420248 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a2a605-4b84-4f43-85f3-0594ad1703fc None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:05.426967 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dcae81-3628-4e20-9313-7c79e8c30fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:05.427541 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dcae81-3628-4e20-9313-7c79e8c30fdd admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:05.428268 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dcae81-3628-4e20-9313-7c79e8c30fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:05.428268 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dcae81-3628-4e20-9313-7c79e8c30fdd admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:05.429254 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dcae81-3628-4e20-9313-7c79e8c30fdd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:33:05.429592 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dcae81-3628-4e20-9313-7c79e8c30fdd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:05.431272 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dcae81-3628-4e20-9313-7c79e8c30fdd admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:05.457772 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b8c68257-1694-4017-9e18-63e09727d6d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e19db6c591fe483d8597643a3f695cf6', 'request_id': 'req-b8c68257-1694-4017-9e18-63e09727d6d8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:05.465412 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1027/2012] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:33:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:33:05.466544 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b8c68257-1694-4017-9e18-63e09727d6d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e19db6c591fe483d8597643a3f695cf6', 'request_id': 'req-b8c68257-1694-4017-9e18-63e09727d6d8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:05.467324 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dcae81-3628-4e20-9313-7c79e8c30fdd admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:05.477022 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b8c68257-1694-4017-9e18-63e09727d6d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e19db6c591fe483d8597643a3f695cf6', 'request_id': 'req-b8c68257-1694-4017-9e18-63e09727d6d8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:05.485215 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dcae81-3628-4e20-9313-7c79e8c30fdd admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:05.489810 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b8c68257-1694-4017-9e18-63e09727d6d8 admin admin] The token cache is being invalidate because project e19db6c591fe483d8597643a3f695cf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:33:05.490247 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b8c68257-1694-4017-9e18-63e09727d6d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:05.490958 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b8c68257-1694-4017-9e18-63e09727d6d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:05.492754 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 986/2013] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:33:05 2022] DELETE /identity/v3/projects/e19db6c591fe483d8597643a3f695cf6 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:05.497935 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c68257-1694-4017-9e18-63e09727d6d8 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:05.499426 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b8c68257-1694-4017-9e18-63e09727d6d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:05.554387 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:05.597697 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce1aa1-4b7d-4f93-9f91-53e9242e9361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:05.599014 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce1aa1-4b7d-4f93-9f91-53e9242e9361 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:05.599164 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce1aa1-4b7d-4f93-9f91-53e9242e9361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:05.599538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce1aa1-4b7d-4f93-9f91-53e9242e9361 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:05.600365 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce1aa1-4b7d-4f93-9f91-53e9242e9361 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:05.602529 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce1aa1-4b7d-4f93-9f91-53e9242e9361 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:05.614043 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c68257-1694-4017-9e18-63e09727d6d8 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:05.628775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e84ecb2-dbb4-4d97-bcc7-49c2a079747a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:05.630061 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e84ecb2-dbb4-4d97-bcc7-49c2a079747a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:05.630410 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e84ecb2-dbb4-4d97-bcc7-49c2a079747a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:05.630771 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e84ecb2-dbb4-4d97-bcc7-49c2a079747a admin admin] PATH_INFO: `/v3/users/bff9f279402d45f18c19624ffb807ce4` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:05.632322 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e84ecb2-dbb4-4d97-bcc7-49c2a079747a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bff9f279402d45f18c19624ffb807ce4)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:05.635796 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e84ecb2-dbb4-4d97-bcc7-49c2a079747a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:05.651784 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1028/2014] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:33:05 2022] POST /identity/v3/projects => generated 355 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:33:05.661533 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce1aa1-4b7d-4f93-9f91-53e9242e9361 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:05.664554 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-69ce1aa1-4b7d-4f93-9f91-53e9242e9361 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:05.718209 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2e84ecb2-dbb4-4d97-bcc7-49c2a079747a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bff9f279402d45f18c19624ffb807ce4'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:05.718920 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2e84ecb2-dbb4-4d97-bcc7-49c2a079747a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bff9f279402d45f18c19624ffb807ce4'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:05.739315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2e84ecb2-dbb4-4d97-bcc7-49c2a079747a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bff9f279402d45f18c19624ffb807ce4'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:05.749915 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2e84ecb2-dbb4-4d97-bcc7-49c2a079747a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bff9f279402d45f18c19624ffb807ce4'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:05.764362 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:05.780584 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2e84ecb2-dbb4-4d97-bcc7-49c2a079747a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bff9f279402d45f18c19624ffb807ce4'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:05.781843 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce1aa1-4b7d-4f93-9f91-53e9242e9361 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:05.788082 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c2dee-db39-45b3-8bf4-3506ca489785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:05.788909 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c2dee-db39-45b3-8bf4-3506ca489785 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:05.789289 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c2dee-db39-45b3-8bf4-3506ca489785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:05.789735 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c2dee-db39-45b3-8bf4-3506ca489785 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:05.790750 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822c2dee-db39-45b3-8bf4-3506ca489785 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:05.795741 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822c2dee-db39-45b3-8bf4-3506ca489785 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:05.804844 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 987/2015] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:33:05 2022] DELETE /identity/v3/users/bff9f279402d45f18c19624ffb807ce4 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:05.805999 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e84ecb2-dbb4-4d97-bcc7-49c2a079747a admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:05.807588 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-2e84ecb2-dbb4-4d97-bcc7-49c2a079747a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:05.824048 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1029/2016] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:33:05 2022] POST /identity/v3/users => generated 278 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:33:05.840656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c2dee-db39-45b3-8bf4-3506ca489785 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:05.843585 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-822c2dee-db39-45b3-8bf4-3506ca489785 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:05.883809 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c2dee-db39-45b3-8bf4-3506ca489785 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:05.931229 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e84ecb2-dbb4-4d97-bcc7-49c2a079747a admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:05.937422 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd7287-5691-4d17-a9f8-b16c96ca3254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:05.938394 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd7287-5691-4d17-a9f8-b16c96ca3254 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:05.938742 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd7287-5691-4d17-a9f8-b16c96ca3254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:05.939047 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd7287-5691-4d17-a9f8-b16c96ca3254 admin admin] PATH_INFO: `/v3/projects/c7f22a04dff547b989a511628fcdf292` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:05.940416 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd7287-5691-4d17-a9f8-b16c96ca3254 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7f22a04dff547b989a511628fcdf292)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:05.942455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd7287-5691-4d17-a9f8-b16c96ca3254 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:05.953134 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674ee2ab-bd3d-4f56-8151-80905188b3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:05.954574 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ee2ab-bd3d-4f56-8151-80905188b3ad admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:05.954885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ee2ab-bd3d-4f56-8151-80905188b3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:05.955147 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ee2ab-bd3d-4f56-8151-80905188b3ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:05.956314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674ee2ab-bd3d-4f56-8151-80905188b3ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:05.957908 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674ee2ab-bd3d-4f56-8151-80905188b3ad admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:05.974289 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1030/2017] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:33:05.977383 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674ee2ab-bd3d-4f56-8151-80905188b3ad admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:05.994840 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674ee2ab-bd3d-4f56-8151-80905188b3ad admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:06.026925 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c6dd7287-5691-4d17-a9f8-b16c96ca3254 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7f22a04dff547b989a511628fcdf292', 'request_id': 'req-c6dd7287-5691-4d17-a9f8-b16c96ca3254'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:06.035222 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c6dd7287-5691-4d17-a9f8-b16c96ca3254 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7f22a04dff547b989a511628fcdf292', 'request_id': 'req-c6dd7287-5691-4d17-a9f8-b16c96ca3254'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:06.047846 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c6dd7287-5691-4d17-a9f8-b16c96ca3254 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7f22a04dff547b989a511628fcdf292', 'request_id': 'req-c6dd7287-5691-4d17-a9f8-b16c96ca3254'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:06.052576 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:06.071536 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c6dd7287-5691-4d17-a9f8-b16c96ca3254 admin admin] The token cache is being invalidate because project c7f22a04dff547b989a511628fcdf292 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:33:06.072018 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c6dd7287-5691-4d17-a9f8-b16c96ca3254 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:06.072738 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c6dd7287-5691-4d17-a9f8-b16c96ca3254 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:06.074522 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 988/2018] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:33:05 2022] DELETE /identity/v3/projects/c7f22a04dff547b989a511628fcdf292 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:06.077711 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95058d2e-6484-4930-940c-f16168bf10b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:06.078342 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd7287-5691-4d17-a9f8-b16c96ca3254 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:06.078918 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95058d2e-6484-4930-940c-f16168bf10b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:06.079333 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95058d2e-6484-4930-940c-f16168bf10b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:06.079886 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95058d2e-6484-4930-940c-f16168bf10b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:06.080140 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c6dd7287-5691-4d17-a9f8-b16c96ca3254 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:06.080947 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95058d2e-6484-4930-940c-f16168bf10b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:06.083095 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95058d2e-6484-4930-940c-f16168bf10b4 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:06.095265 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd7287-5691-4d17-a9f8-b16c96ca3254 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:06.097944 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1031/2019] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:33:06.102160 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95058d2e-6484-4930-940c-f16168bf10b4 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:06.104275 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-95058d2e-6484-4930-940c-f16168bf10b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:06.111073 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7075ef3f-1a18-40b7-9fe5-6f72e78a59da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:06.113393 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075ef3f-1a18-40b7-9fe5-6f72e78a59da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:06.113580 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075ef3f-1a18-40b7-9fe5-6f72e78a59da admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:06.114614 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075ef3f-1a18-40b7-9fe5-6f72e78a59da admin admin] PATH_INFO: `/v3/users/8c3933fd54294c00b09738b92db89b7a` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:06.115381 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7075ef3f-1a18-40b7-9fe5-6f72e78a59da admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c3933fd54294c00b09738b92db89b7a)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:06.118714 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7075ef3f-1a18-40b7-9fe5-6f72e78a59da admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:06.119638 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95058d2e-6484-4930-940c-f16168bf10b4 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:06.126385 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acdc098-9e61-4d0a-9f48-69af4ea05a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:06.127377 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdc098-9e61-4d0a-9f48-69af4ea05a75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:06.131607 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdc098-9e61-4d0a-9f48-69af4ea05a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:06.131607 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdc098-9e61-4d0a-9f48-69af4ea05a75 admin admin] PATH_INFO: `/v3/projects/05c47588614d40cca446f0346af9543f/users/8404904966c84007a5f39ca6f58b10dd/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:06.157351 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acdc098-9e61-4d0a-9f48-69af4ea05a75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05c47588614d40cca446f0346af9543f, user_id=8404904966c84007a5f39ca6f58b10dd, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:06.162682 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acdc098-9e61-4d0a-9f48-69af4ea05a75 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:06.187105 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1032/2020] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:33:06 2022] PUT /identity/v3/projects/05c47588614d40cca446f0346af9543f/users/8404904966c84007a5f39ca6f58b10dd/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:06.199638 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c1452-04f4-4c6e-a517-162a41dea865 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:06.199976 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c1452-04f4-4c6e-a517-162a41dea865 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:06.200320 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c1452-04f4-4c6e-a517-162a41dea865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:06.218275 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7075ef3f-1a18-40b7-9fe5-6f72e78a59da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c3933fd54294c00b09738b92db89b7a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:06.220768 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7075ef3f-1a18-40b7-9fe5-6f72e78a59da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c3933fd54294c00b09738b92db89b7a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:06.261126 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7075ef3f-1a18-40b7-9fe5-6f72e78a59da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c3933fd54294c00b09738b92db89b7a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:06.267132 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7075ef3f-1a18-40b7-9fe5-6f72e78a59da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c3933fd54294c00b09738b92db89b7a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:06.268610 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-632c1452-04f4-4c6e-a517-162a41dea865 None None] MFA Rules not processed for user `8404904966c84007a5f39ca6f58b10dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:06.270068 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-632c1452-04f4-4c6e-a517-162a41dea865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:06.280323 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7075ef3f-1a18-40b7-9fe5-6f72e78a59da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c3933fd54294c00b09738b92db89b7a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:06.285563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:06.310055 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 989/2021] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:33:06 2022] DELETE /identity/v3/users/8c3933fd54294c00b09738b92db89b7a => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:33:06.310055 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7075ef3f-1a18-40b7-9fe5-6f72e78a59da admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:06.310055 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7075ef3f-1a18-40b7-9fe5-6f72e78a59da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:06.464579 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1033/2022] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:06 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:06.469208 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c1452-04f4-4c6e-a517-162a41dea865 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:06.472399 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-632c1452-04f4-4c6e-a517-162a41dea865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:06.472693 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7075ef3f-1a18-40b7-9fe5-6f72e78a59da admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:06.484422 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f75f5da-0840-4390-b6f6-a03742e213e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:06.486050 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75f5da-0840-4390-b6f6-a03742e213e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:06.486050 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75f5da-0840-4390-b6f6-a03742e213e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:06.490311 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75f5da-0840-4390-b6f6-a03742e213e9 admin admin] PATH_INFO: `/v3/projects/69392f9b92f5411eb44fb644d126e31f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:06.494403 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f75f5da-0840-4390-b6f6-a03742e213e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69392f9b92f5411eb44fb644d126e31f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:06.497361 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f75f5da-0840-4390-b6f6-a03742e213e9 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:06.564087 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9f75f5da-0840-4390-b6f6-a03742e213e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69392f9b92f5411eb44fb644d126e31f', 'request_id': 'req-9f75f5da-0840-4390-b6f6-a03742e213e9'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:06.571236 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9f75f5da-0840-4390-b6f6-a03742e213e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69392f9b92f5411eb44fb644d126e31f', 'request_id': 'req-9f75f5da-0840-4390-b6f6-a03742e213e9'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:06.579276 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c1452-04f4-4c6e-a517-162a41dea865 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:06.584929 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9f75f5da-0840-4390-b6f6-a03742e213e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69392f9b92f5411eb44fb644d126e31f', 'request_id': 'req-9f75f5da-0840-4390-b6f6-a03742e213e9'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:06.605476 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9f75f5da-0840-4390-b6f6-a03742e213e9 admin admin] The token cache is being invalidate because project 69392f9b92f5411eb44fb644d126e31f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:33:06.605765 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9f75f5da-0840-4390-b6f6-a03742e213e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:06.607222 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9f75f5da-0840-4390-b6f6-a03742e213e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:06.612963 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 990/2023] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:33:06 2022] DELETE /identity/v3/projects/69392f9b92f5411eb44fb644d126e31f => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:06.615568 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eab3bf-1586-43af-a189-5de66e7b4986 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:06.615844 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eab3bf-1586-43af-a189-5de66e7b4986 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:06.616128 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eab3bf-1586-43af-a189-5de66e7b4986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:06.653885 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-89eab3bf-1586-43af-a189-5de66e7b4986 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:06.655424 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-89eab3bf-1586-43af-a189-5de66e7b4986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:06.680989 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:06.696522 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b8fee-c8f0-4e8d-888f-73cd770d600e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:06.697213 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b8fee-c8f0-4e8d-888f-73cd770d600e admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:06.697539 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b8fee-c8f0-4e8d-888f-73cd770d600e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:06.697815 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b8fee-c8f0-4e8d-888f-73cd770d600e admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:06.698560 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b8fee-c8f0-4e8d-888f-73cd770d600e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:06.701259 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b8fee-c8f0-4e8d-888f-73cd770d600e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:06.745563 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1034/2024] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:33:06 2022] POST /identity/v3/projects => generated 357 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:33:06.749772 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b8fee-c8f0-4e8d-888f-73cd770d600e admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:06.751828 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-de2b8fee-c8f0-4e8d-888f-73cd770d600e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:06.771728 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 991/2025] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:06.781159 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb118f-acc8-4fc6-89d8-779e4205d75e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:06.781451 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb118f-acc8-4fc6-89d8-779e4205d75e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:06.781737 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb118f-acc8-4fc6-89d8-779e4205d75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:06.813170 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-bffb118f-acc8-4fc6-89d8-779e4205d75e None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:06.815738 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-bffb118f-acc8-4fc6-89d8-779e4205d75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:06.847580 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b8fee-c8f0-4e8d-888f-73cd770d600e admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:06.856780 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:06.871900 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57d26b-74f6-4cba-a399-72d7fb6c3c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:06.872737 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57d26b-74f6-4cba-a399-72d7fb6c3c9b admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:06.873121 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57d26b-74f6-4cba-a399-72d7fb6c3c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:06.873464 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57d26b-74f6-4cba-a399-72d7fb6c3c9b admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:06.874368 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c57d26b-74f6-4cba-a399-72d7fb6c3c9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:06.877004 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c57d26b-74f6-4cba-a399-72d7fb6c3c9b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:06.905374 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1035/2026] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:33:06 2022] POST /identity/v3/users => generated 285 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:33:06.906372 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57d26b-74f6-4cba-a399-72d7fb6c3c9b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:06.908253 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3c57d26b-74f6-4cba-a399-72d7fb6c3c9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:06.924361 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57d26b-74f6-4cba-a399-72d7fb6c3c9b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:06.928787 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 992/2027] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:33:06.929716 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffb118f-acc8-4fc6-89d8-779e4205d75e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:06.932817 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf9e33-8213-46d8-aea6-382886ff5504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:06.934570 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf9e33-8213-46d8-aea6-382886ff5504 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:06.934570 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf9e33-8213-46d8-aea6-382886ff5504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:06.934570 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf9e33-8213-46d8-aea6-382886ff5504 admin admin] PATH_INFO: `/v3/projects/942fed1f8cbc4499a16bd2199a027a0f` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:06.935645 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bf9e33-8213-46d8-aea6-382886ff5504 admin admin] RBAC: Authorizing `identity:delete_project(project_id=942fed1f8cbc4499a16bd2199a027a0f)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:06.937609 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bf9e33-8213-46d8-aea6-382886ff5504 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:06.941741 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffb118f-acc8-4fc6-89d8-779e4205d75e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:06.951024 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1bff55-9977-4ff0-9a12-c208a64a1eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:06.952058 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1bff55-9977-4ff0-9a12-c208a64a1eb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:06.952532 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1bff55-9977-4ff0-9a12-c208a64a1eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:06.952901 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1bff55-9977-4ff0-9a12-c208a64a1eb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:06.954092 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1bff55-9977-4ff0-9a12-c208a64a1eb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:06.956534 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1bff55-9977-4ff0-9a12-c208a64a1eb6 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:06.985391 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 993/2028] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:33:06.987240 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1bff55-9977-4ff0-9a12-c208a64a1eb6 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:07.004121 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1bff55-9977-4ff0-9a12-c208a64a1eb6 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:07.009241 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-57bf9e33-8213-46d8-aea6-382886ff5504 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '942fed1f8cbc4499a16bd2199a027a0f', 'request_id': 'req-57bf9e33-8213-46d8-aea6-382886ff5504'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:07.018343 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-57bf9e33-8213-46d8-aea6-382886ff5504 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '942fed1f8cbc4499a16bd2199a027a0f', 'request_id': 'req-57bf9e33-8213-46d8-aea6-382886ff5504'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:07.026394 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-57bf9e33-8213-46d8-aea6-382886ff5504 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '942fed1f8cbc4499a16bd2199a027a0f', 'request_id': 'req-57bf9e33-8213-46d8-aea6-382886ff5504'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:07.062430 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-57bf9e33-8213-46d8-aea6-382886ff5504 admin admin] The token cache is being invalidate because project 942fed1f8cbc4499a16bd2199a027a0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:33:07.062944 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-57bf9e33-8213-46d8-aea6-382886ff5504 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:07.063790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-57bf9e33-8213-46d8-aea6-382886ff5504 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:07.068635 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1036/2029] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:33:06 2022] DELETE /identity/v3/projects/942fed1f8cbc4499a16bd2199a027a0f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:07.071501 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf9e33-8213-46d8-aea6-382886ff5504 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:07.073370 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-57bf9e33-8213-46d8-aea6-382886ff5504 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:07.091403 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb926728-9459-43e5-a4b2-e45e6d5885b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:07.092094 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb926728-9459-43e5-a4b2-e45e6d5885b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:07.092582 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb926728-9459-43e5-a4b2-e45e6d5885b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:07.093632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb926728-9459-43e5-a4b2-e45e6d5885b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:07.094581 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb926728-9459-43e5-a4b2-e45e6d5885b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:33:07.095218 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb926728-9459-43e5-a4b2-e45e6d5885b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:07.096936 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb926728-9459-43e5-a4b2-e45e6d5885b8 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:07.131748 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 994/2030] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:33:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:33:07.131748 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53223b87-8cbf-4915-ab59-1812959d639c None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:07.131748 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53223b87-8cbf-4915-ab59-1812959d639c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:07.131748 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53223b87-8cbf-4915-ab59-1812959d639c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:07.147037 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:07.154898 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-53223b87-8cbf-4915-ab59-1812959d639c None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:07.156117 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-53223b87-8cbf-4915-ab59-1812959d639c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:07.163710 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf9e33-8213-46d8-aea6-382886ff5504 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:07.165198 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 995/2031] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:07.165884 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53223b87-8cbf-4915-ab59-1812959d639c None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:07.168864 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720e209-5d77-40c3-8797-07a66ff155ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:07.169817 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720e209-5d77-40c3-8797-07a66ff155ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:07.170156 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720e209-5d77-40c3-8797-07a66ff155ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:07.170467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720e209-5d77-40c3-8797-07a66ff155ef admin admin] PATH_INFO: `/v3/projects/73c9075e9c4844d798e959056611266b/users/857d05e10de645368a4cfe5af49c13cc/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:07.173790 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-53223b87-8cbf-4915-ab59-1812959d639c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:07.186941 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53223b87-8cbf-4915-ab59-1812959d639c None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:07.189423 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720e209-5d77-40c3-8797-07a66ff155ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=73c9075e9c4844d798e959056611266b, user_id=857d05e10de645368a4cfe5af49c13cc, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:07.192038 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee4fff-bfed-4b66-874a-5c4b66e18b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:07.192371 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720e209-5d77-40c3-8797-07a66ff155ef admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:07.192718 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee4fff-bfed-4b66-874a-5c4b66e18b44 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:07.193007 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee4fff-bfed-4b66-874a-5c4b66e18b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:07.193285 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee4fff-bfed-4b66-874a-5c4b66e18b44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:07.193998 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eee4fff-bfed-4b66-874a-5c4b66e18b44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:07.195982 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eee4fff-bfed-4b66-874a-5c4b66e18b44 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:07.207859 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1037/2032] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:33:07 2022] PUT /identity/v3/projects/73c9075e9c4844d798e959056611266b/users/857d05e10de645368a4cfe5af49c13cc/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:07.209470 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16141642-8bcd-4724-b363-eb7fcdefffc7 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:07.209776 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16141642-8bcd-4724-b363-eb7fcdefffc7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:07.210044 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16141642-8bcd-4724-b363-eb7fcdefffc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:07.229862 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-16141642-8bcd-4724-b363-eb7fcdefffc7 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:07.231176 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-16141642-8bcd-4724-b363-eb7fcdefffc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:07.239343 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 996/2033] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:33:07 2022] POST /identity/v3/projects => generated 375 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:33:07.246294 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee4fff-bfed-4b66-874a-5c4b66e18b44 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:07.338819 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1038/2034] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:07.340894 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16141642-8bcd-4724-b363-eb7fcdefffc7 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:07.351981 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee4fff-bfed-4b66-874a-5c4b66e18b44 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:07.353886 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16141642-8bcd-4724-b363-eb7fcdefffc7 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:07.359916 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b090d2-9b2e-48bc-b28c-93ab61418cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:07.360734 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b090d2-9b2e-48bc-b28c-93ab61418cdc admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:07.361260 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b090d2-9b2e-48bc-b28c-93ab61418cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:07.361664 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b090d2-9b2e-48bc-b28c-93ab61418cdc admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:07.362735 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d7fb91-eb86-46cf-8c69-0dec9b8c402b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:07.362885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b090d2-9b2e-48bc-b28c-93ab61418cdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:07.363526 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7fb91-eb86-46cf-8c69-0dec9b8c402b admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:07.363876 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7fb91-eb86-46cf-8c69-0dec9b8c402b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:07.365498 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7fb91-eb86-46cf-8c69-0dec9b8c402b admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:07.365498 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d7fb91-eb86-46cf-8c69-0dec9b8c402b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:07.365715 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b090d2-9b2e-48bc-b28c-93ab61418cdc admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:07.366776 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d7fb91-eb86-46cf-8c69-0dec9b8c402b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:07.381721 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 997/2035] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:33:07.382790 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d7fb91-eb86-46cf-8c69-0dec9b8c402b admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:07.395113 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d7fb91-eb86-46cf-8c69-0dec9b8c402b admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:07.403529 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1039/2036] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:33:07 2022] POST /identity/v3/users => generated 288 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:33:07.404243 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a049d72-deb6-4096-a2ed-d3979f612b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:07.404461 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b090d2-9b2e-48bc-b28c-93ab61418cdc admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:07.405384 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a049d72-deb6-4096-a2ed-d3979f612b5e admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:07.405668 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a049d72-deb6-4096-a2ed-d3979f612b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:07.406137 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a049d72-deb6-4096-a2ed-d3979f612b5e admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:07.406942 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a049d72-deb6-4096-a2ed-d3979f612b5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:33:07.407280 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a049d72-deb6-4096-a2ed-d3979f612b5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:07.409091 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a049d72-deb6-4096-a2ed-d3979f612b5e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:07.417319 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b090d2-9b2e-48bc-b28c-93ab61418cdc admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:07.423007 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b8570-2e08-43a6-99a0-492485ea1c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:07.423195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b8570-2e08-43a6-99a0-492485ea1c2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:07.423460 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b8570-2e08-43a6-99a0-492485ea1c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:07.423746 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b8570-2e08-43a6-99a0-492485ea1c2f admin admin] PATH_INFO: `/v3/projects/73c9075e9c4844d798e959056611266b/users/857d05e10de645368a4cfe5af49c13cc/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:07.426945 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063b8570-2e08-43a6-99a0-492485ea1c2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=73c9075e9c4844d798e959056611266b, user_id=857d05e10de645368a4cfe5af49c13cc, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:07.428054 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 998/2037] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:33:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:33:07.429396 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a049d72-deb6-4096-a2ed-d3979f612b5e admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:07.430218 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063b8570-2e08-43a6-99a0-492485ea1c2f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:07.442186 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a049d72-deb6-4096-a2ed-d3979f612b5e admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:07.447421 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1040/2038] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:33:07 2022] PUT /identity/v3/projects/73c9075e9c4844d798e959056611266b/users/857d05e10de645368a4cfe5af49c13cc/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:07.448352 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:07.448488 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b8570-2e08-43a6-99a0-492485ea1c2f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:07.535574 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b8570-2e08-43a6-99a0-492485ea1c2f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:07.538518 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf1067-bd1f-43cc-9114-bc6afa7ef6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:07.539188 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf1067-bd1f-43cc-9114-bc6afa7ef6b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:07.539479 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf1067-bd1f-43cc-9114-bc6afa7ef6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:07.539757 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf1067-bd1f-43cc-9114-bc6afa7ef6b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:07.541060 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccf1067-bd1f-43cc-9114-bc6afa7ef6b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:07.541870 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f4acd-6dbe-41ff-8036-3e0c65aee5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:07.543351 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f4acd-6dbe-41ff-8036-3e0c65aee5ad admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:07.543470 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccf1067-bd1f-43cc-9114-bc6afa7ef6b3 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:07.543547 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f4acd-6dbe-41ff-8036-3e0c65aee5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:07.543547 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f4acd-6dbe-41ff-8036-3e0c65aee5ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:07.543988 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52f4acd-6dbe-41ff-8036-3e0c65aee5ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:07.546117 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52f4acd-6dbe-41ff-8036-3e0c65aee5ad admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:07.557075 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 999/2039] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:33:07.557075 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcaceb3-fc48-4e35-9a6e-6969773ac94b None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:07.557075 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcaceb3-fc48-4e35-9a6e-6969773ac94b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:07.557075 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcaceb3-fc48-4e35-9a6e-6969773ac94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:07.576699 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1041/2040] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:33:07 2022] POST /identity/v3/projects => generated 359 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:33:07.577837 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f4acd-6dbe-41ff-8036-3e0c65aee5ad admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:07.587612 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-1bcaceb3-fc48-4e35-9a6e-6969773ac94b None None] MFA Rules not processed for user `857d05e10de645368a4cfe5af49c13cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:07.589555 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1bcaceb3-fc48-4e35-9a6e-6969773ac94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:07.635763 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:07.676914 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1000/2041] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:07 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:07.679321 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcaceb3-fc48-4e35-9a6e-6969773ac94b None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:07.680368 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f4acd-6dbe-41ff-8036-3e0c65aee5ad admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:07.686128 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee75f5-ec36-4a5d-a448-3a7db4a0532b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:07.686953 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee75f5-ec36-4a5d-a448-3a7db4a0532b admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:07.687189 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee75f5-ec36-4a5d-a448-3a7db4a0532b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:07.687449 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee75f5-ec36-4a5d-a448-3a7db4a0532b admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:07.688171 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee75f5-ec36-4a5d-a448-3a7db4a0532b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:07.689854 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee75f5-ec36-4a5d-a448-3a7db4a0532b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:07.693434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcaceb3-fc48-4e35-9a6e-6969773ac94b None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:07.698178 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0bbd92-b43c-4f59-af30-bd0e7400ed7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:07.698887 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0bbd92-b43c-4f59-af30-bd0e7400ed7e admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:07.699168 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0bbd92-b43c-4f59-af30-bd0e7400ed7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:07.699447 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0bbd92-b43c-4f59-af30-bd0e7400ed7e admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:07.700134 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0bbd92-b43c-4f59-af30-bd0e7400ed7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:07.703806 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1042/2042] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:33:07.703806 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee75f5-ec36-4a5d-a448-3a7db4a0532b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:07.704030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0bbd92-b43c-4f59-af30-bd0e7400ed7e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:07.704105 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-86ee75f5-ec36-4a5d-a448-3a7db4a0532b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:07.733403 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1001/2043] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:33:07 2022] POST /identity/v3/users => generated 280 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:33:07.734614 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0bbd92-b43c-4f59-af30-bd0e7400ed7e admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:07.746629 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0bbd92-b43c-4f59-af30-bd0e7400ed7e admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:07.752713 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512e2065-8aec-4fc3-9781-711f7162db21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:07.753962 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e2065-8aec-4fc3-9781-711f7162db21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:07.754448 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e2065-8aec-4fc3-9781-711f7162db21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:07.754732 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e2065-8aec-4fc3-9781-711f7162db21 admin admin] PATH_INFO: `/v3/projects/dd923b1d88e141b4b4748fff9395b46f/users/02d73e3baf9e434fabb2785ade49fd51/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:07.775540 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512e2065-8aec-4fc3-9781-711f7162db21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd923b1d88e141b4b4748fff9395b46f, user_id=02d73e3baf9e434fabb2785ade49fd51, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:07.776106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512e2065-8aec-4fc3-9781-711f7162db21 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:07.782359 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee75f5-ec36-4a5d-a448-3a7db4a0532b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:07.787145 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc43ac6b-99a3-4de4-9b80-4e8f02d5c9f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:07.788112 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43ac6b-99a3-4de4-9b80-4e8f02d5c9f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:07.788543 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43ac6b-99a3-4de4-9b80-4e8f02d5c9f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:07.788850 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43ac6b-99a3-4de4-9b80-4e8f02d5c9f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:07.791827 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1002/2044] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:33:07 2022] PUT /identity/v3/projects/dd923b1d88e141b4b4748fff9395b46f/users/02d73e3baf9e434fabb2785ade49fd51/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:07.792906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512e2065-8aec-4fc3-9781-711f7162db21 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:07.801038 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc43ac6b-99a3-4de4-9b80-4e8f02d5c9f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:07.803442 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc43ac6b-99a3-4de4-9b80-4e8f02d5c9f0 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:07.814413 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1043/2045] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:33:07 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:33:07.816006 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa53e2-2eb2-4fdc-9588-6a581f058b43 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:07.816362 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa53e2-2eb2-4fdc-9588-6a581f058b43 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:07.817088 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa53e2-2eb2-4fdc-9588-6a581f058b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:07.847368 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c7fa53e2-2eb2-4fdc-9588-6a581f058b43 None None] MFA Rules not processed for user `02d73e3baf9e434fabb2785ade49fd51`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:07.848678 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c7fa53e2-2eb2-4fdc-9588-6a581f058b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:07.855945 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:07.886427 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512e2065-8aec-4fc3-9781-711f7162db21 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:07.891918 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121c6c29-a6d2-482b-9791-7bf36699e8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:07.893582 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c6c29-a6d2-482b-9791-7bf36699e8a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:07.894027 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c6c29-a6d2-482b-9791-7bf36699e8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:07.894611 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c6c29-a6d2-482b-9791-7bf36699e8a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:07.895515 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121c6c29-a6d2-482b-9791-7bf36699e8a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:07.898700 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121c6c29-a6d2-482b-9791-7bf36699e8a3 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:07.912837 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1003/2046] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:33:07.916992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121c6c29-a6d2-482b-9791-7bf36699e8a3 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:07.938878 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1044/2047] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:07.952584 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121c6c29-a6d2-482b-9791-7bf36699e8a3 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:07.956006 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa53e2-2eb2-4fdc-9588-6a581f058b43 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:07.957976 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c7fa53e2-2eb2-4fdc-9588-6a581f058b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:07.960331 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc02ea0-f1c6-4182-b607-14f17a0761af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:07.961131 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc02ea0-f1c6-4182-b607-14f17a0761af admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:07.961517 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc02ea0-f1c6-4182-b607-14f17a0761af admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:07.961959 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc02ea0-f1c6-4182-b607-14f17a0761af admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:07.962832 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc02ea0-f1c6-4182-b607-14f17a0761af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:07.964558 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc02ea0-f1c6-4182-b607-14f17a0761af admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:07.977464 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1004/2048] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:33:07.978532 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc02ea0-f1c6-4182-b607-14f17a0761af admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:08.025516 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa53e2-2eb2-4fdc-9588-6a581f058b43 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:08.029289 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed47422-21ac-4e34-b022-2a23c6b1e536 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:08.029946 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed47422-21ac-4e34-b022-2a23c6b1e536 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:08.030228 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed47422-21ac-4e34-b022-2a23c6b1e536 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:08.030527 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed47422-21ac-4e34-b022-2a23c6b1e536 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:08.038041 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc02ea0-f1c6-4182-b607-14f17a0761af admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:08.039555 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed47422-21ac-4e34-b022-2a23c6b1e536 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:08.041007 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed47422-21ac-4e34-b022-2a23c6b1e536 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:08.042435 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d450a-11f2-467d-9ff3-e1dcd646e2aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:08.043100 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d450a-11f2-467d-9ff3-e1dcd646e2aa service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:08.043388 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d450a-11f2-467d-9ff3-e1dcd646e2aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:08.043669 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d450a-11f2-467d-9ff3-e1dcd646e2aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:08.044808 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:08.046907 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-250d450a-11f2-467d-9ff3-e1dcd646e2aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:08.055266 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1045/2049] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:33:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:33:08.056354 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed47422-21ac-4e34-b022-2a23c6b1e536 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:08.069928 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed47422-21ac-4e34-b022-2a23c6b1e536 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:08.076050 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663f3213-102a-4697-91b4-9f989ef39f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:08.076949 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f3213-102a-4697-91b4-9f989ef39f3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:08.077650 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f3213-102a-4697-91b4-9f989ef39f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:08.077924 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f3213-102a-4697-91b4-9f989ef39f3f admin admin] PATH_INFO: `/v3/projects/3d626f40ae254b4a82d76f9e191b8c0a/users/1b043847595b45058d6cd418ee41a6bc/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:08.096959 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663f3213-102a-4697-91b4-9f989ef39f3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d626f40ae254b4a82d76f9e191b8c0a, user_id=1b043847595b45058d6cd418ee41a6bc, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:08.099657 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663f3213-102a-4697-91b4-9f989ef39f3f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:08.117973 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d450a-11f2-467d-9ff3-e1dcd646e2aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:08.119850 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1046/2050] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:33:08 2022] PUT /identity/v3/projects/3d626f40ae254b4a82d76f9e191b8c0a/users/1b043847595b45058d6cd418ee41a6bc/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:08.119970 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d450a-11f2-467d-9ff3-e1dcd646e2aa service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:08.130369 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b13b1d-c868-4aa4-8509-5fc8c3bb11bd None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:08.130660 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b13b1d-c868-4aa4-8509-5fc8c3bb11bd None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:08.130935 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b13b1d-c868-4aa4-8509-5fc8c3bb11bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:08.132062 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1005/2051] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:33:07 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:33:08.210101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-78b13b1d-c868-4aa4-8509-5fc8c3bb11bd None None] MFA Rules not processed for user `1b043847595b45058d6cd418ee41a6bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:08.214766 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-78b13b1d-c868-4aa4-8509-5fc8c3bb11bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:08.400103 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1047/2052] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:08 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:33:08.544715 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d450a-11f2-467d-9ff3-e1dcd646e2aa service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:08.559690 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d450a-11f2-467d-9ff3-e1dcd646e2aa service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:08.656963 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f373f-0fac-4308-9c17-dbd47dd892a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:08.658182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f373f-0fac-4308-9c17-dbd47dd892a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:08.658742 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f373f-0fac-4308-9c17-dbd47dd892a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:08.659077 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f373f-0fac-4308-9c17-dbd47dd892a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:08.670976 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f373f-0fac-4308-9c17-dbd47dd892a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:08.672462 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f373f-0fac-4308-9c17-dbd47dd892a3 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:08.686787 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1006/2053] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:33:08 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:33:08.702759 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b13b1d-c868-4aa4-8509-5fc8c3bb11bd None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:08.780307 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b13b1d-c868-4aa4-8509-5fc8c3bb11bd None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:08.786291 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf6586-ef86-4a3b-8d1d-c20bf15676a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:08.787012 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf6586-ef86-4a3b-8d1d-c20bf15676a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:08.787211 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf6586-ef86-4a3b-8d1d-c20bf15676a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:08.787529 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf6586-ef86-4a3b-8d1d-c20bf15676a4 admin admin] PATH_INFO: `/v3/users/1b043847595b45058d6cd418ee41a6bc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:08.788965 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bf6586-ef86-4a3b-8d1d-c20bf15676a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b043847595b45058d6cd418ee41a6bc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:08.790841 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bf6586-ef86-4a3b-8d1d-c20bf15676a4 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:08.846853 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-85bf6586-ef86-4a3b-8d1d-c20bf15676a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b043847595b45058d6cd418ee41a6bc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:08.860675 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-85bf6586-ef86-4a3b-8d1d-c20bf15676a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b043847595b45058d6cd418ee41a6bc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:08.861695 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-85bf6586-ef86-4a3b-8d1d-c20bf15676a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b043847595b45058d6cd418ee41a6bc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:08.862187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-85bf6586-ef86-4a3b-8d1d-c20bf15676a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b043847595b45058d6cd418ee41a6bc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:08.863536 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:08.871362 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-85bf6586-ef86-4a3b-8d1d-c20bf15676a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b043847595b45058d6cd418ee41a6bc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:08.888118 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1048/2054] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:33:08 2022] DELETE /identity/v3/users/1b043847595b45058d6cd418ee41a6bc => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:08.893354 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c76d3a-5a84-4bd3-bd96-5ff91b2868e4 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:08.893733 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c76d3a-5a84-4bd3-bd96-5ff91b2868e4 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:08.894081 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c76d3a-5a84-4bd3-bd96-5ff91b2868e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:08.926344 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-03c76d3a-5a84-4bd3-bd96-5ff91b2868e4 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:08.928087 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-03c76d3a-5a84-4bd3-bd96-5ff91b2868e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:08.987606 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:09.102120 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1007/2055] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:09.114147 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf6586-ef86-4a3b-8d1d-c20bf15676a4 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:09.115824 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-85bf6586-ef86-4a3b-8d1d-c20bf15676a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:09.193756 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf6586-ef86-4a3b-8d1d-c20bf15676a4 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:09.197765 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce553b-4d5f-4751-a7d9-576055d060b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:09.198690 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce553b-4d5f-4751-a7d9-576055d060b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:09.199182 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce553b-4d5f-4751-a7d9-576055d060b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:09.199464 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce553b-4d5f-4751-a7d9-576055d060b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:09.211798 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce553b-4d5f-4751-a7d9-576055d060b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:09.213266 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce553b-4d5f-4751-a7d9-576055d060b9 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:09.225621 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1049/2056] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:33:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:33:09.703614 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c76d3a-5a84-4bd3-bd96-5ff91b2868e4 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:09.706109 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-03c76d3a-5a84-4bd3-bd96-5ff91b2868e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:09.731868 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c76d3a-5a84-4bd3-bd96-5ff91b2868e4 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:09.779563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad658e-99be-4f5a-bc1b-89b7217e7930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:09.780346 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad658e-99be-4f5a-bc1b-89b7217e7930 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:09.780696 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad658e-99be-4f5a-bc1b-89b7217e7930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:09.780970 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad658e-99be-4f5a-bc1b-89b7217e7930 admin admin] PATH_INFO: `/v3/projects/3d626f40ae254b4a82d76f9e191b8c0a` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:09.782450 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ad658e-99be-4f5a-bc1b-89b7217e7930 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d626f40ae254b4a82d76f9e191b8c0a)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:09.784935 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ad658e-99be-4f5a-bc1b-89b7217e7930 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:09.855119 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-02ad658e-99be-4f5a-bc1b-89b7217e7930 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d626f40ae254b4a82d76f9e191b8c0a', 'request_id': 'req-02ad658e-99be-4f5a-bc1b-89b7217e7930'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:09.865846 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-02ad658e-99be-4f5a-bc1b-89b7217e7930 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d626f40ae254b4a82d76f9e191b8c0a', 'request_id': 'req-02ad658e-99be-4f5a-bc1b-89b7217e7930'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:09.880896 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-02ad658e-99be-4f5a-bc1b-89b7217e7930 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d626f40ae254b4a82d76f9e191b8c0a', 'request_id': 'req-02ad658e-99be-4f5a-bc1b-89b7217e7930'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:09.909297 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-02ad658e-99be-4f5a-bc1b-89b7217e7930 admin admin] The token cache is being invalidate because project 3d626f40ae254b4a82d76f9e191b8c0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:33:09.910034 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-02ad658e-99be-4f5a-bc1b-89b7217e7930 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:09.912196 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-02ad658e-99be-4f5a-bc1b-89b7217e7930 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:09.916848 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1008/2057] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:33:09 2022] DELETE /identity/v3/projects/3d626f40ae254b4a82d76f9e191b8c0a => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:09.924804 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ff74f-6f9c-41e4-8cd0-068bf1dad464 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:09.925267 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ff74f-6f9c-41e4-8cd0-068bf1dad464 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:09.925716 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ff74f-6f9c-41e4-8cd0-068bf1dad464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:09.974339 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-508ff74f-6f9c-41e4-8cd0-068bf1dad464 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:09.976894 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-508ff74f-6f9c-41e4-8cd0-068bf1dad464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:10.158800 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1050/2058] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:10.161899 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68176832-a1a1-40a9-aa04-a6b9154e54ec None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:10.162439 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68176832-a1a1-40a9-aa04-a6b9154e54ec None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:10.167077 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68176832-a1a1-40a9-aa04-a6b9154e54ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:10.204460 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-68176832-a1a1-40a9-aa04-a6b9154e54ec None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:10.207126 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-68176832-a1a1-40a9-aa04-a6b9154e54ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:10.227193 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1009/2059] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:10.227910 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ff74f-6f9c-41e4-8cd0-068bf1dad464 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:10.254927 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ff74f-6f9c-41e4-8cd0-068bf1dad464 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:10.264243 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab43f6-0d0e-42e2-9a4c-ced1bee304b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:10.265618 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab43f6-0d0e-42e2-9a4c-ced1bee304b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:10.266907 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab43f6-0d0e-42e2-9a4c-ced1bee304b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:10.266907 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab43f6-0d0e-42e2-9a4c-ced1bee304b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:10.269302 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fab43f6-0d0e-42e2-9a4c-ced1bee304b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:33:10.269302 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fab43f6-0d0e-42e2-9a4c-ced1bee304b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:10.274008 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fab43f6-0d0e-42e2-9a4c-ced1bee304b3 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:10.309737 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68176832-a1a1-40a9-aa04-a6b9154e54ec None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:10.331618 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1051/2060] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:33:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:33:10.357302 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68176832-a1a1-40a9-aa04-a6b9154e54ec None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:10.363738 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713bacc-5809-40b4-8fe6-b657c93134fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:10.364988 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713bacc-5809-40b4-8fe6-b657c93134fc admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:10.365526 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713bacc-5809-40b4-8fe6-b657c93134fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:10.366094 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713bacc-5809-40b4-8fe6-b657c93134fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:10.366999 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b713bacc-5809-40b4-8fe6-b657c93134fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:10.389761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b713bacc-5809-40b4-8fe6-b657c93134fc admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:10.506352 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1010/2061] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:33:10 2022] POST /identity/v3/projects => generated 367 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:33:10.511505 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab43f6-0d0e-42e2-9a4c-ced1bee304b3 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:10.527253 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab43f6-0d0e-42e2-9a4c-ced1bee304b3 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:10.643863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77510b87-e592-48fc-bb95-a517d20b55e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:10.644756 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77510b87-e592-48fc-bb95-a517d20b55e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:10.645040 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77510b87-e592-48fc-bb95-a517d20b55e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:10.645317 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77510b87-e592-48fc-bb95-a517d20b55e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:10.646084 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77510b87-e592-48fc-bb95-a517d20b55e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:10.648406 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77510b87-e592-48fc-bb95-a517d20b55e9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:10.707175 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1052/2062] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:33:10 2022] POST /identity/v3/users => generated 284 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:33:10.713630 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713bacc-5809-40b4-8fe6-b657c93134fc admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:10.780747 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713bacc-5809-40b4-8fe6-b657c93134fc admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:10.806782 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f8ae4-0298-42aa-9e31-cf338757c03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:10.808145 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f8ae4-0298-42aa-9e31-cf338757c03d admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:10.816292 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f8ae4-0298-42aa-9e31-cf338757c03d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:10.818843 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f8ae4-0298-42aa-9e31-cf338757c03d admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:10.820484 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f8ae4-0298-42aa-9e31-cf338757c03d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:10.823519 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f8ae4-0298-42aa-9e31-cf338757c03d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:10.842142 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1011/2063] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:33:10.854734 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77510b87-e592-48fc-bb95-a517d20b55e9 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:10.877688 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:10.891290 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77510b87-e592-48fc-bb95-a517d20b55e9 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:10.911271 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08432454-af4d-4b81-bb75-d61e0c2672a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:10.912090 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08432454-af4d-4b81-bb75-d61e0c2672a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:10.912490 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08432454-af4d-4b81-bb75-d61e0c2672a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:10.914042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08432454-af4d-4b81-bb75-d61e0c2672a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:10.916675 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08432454-af4d-4b81-bb75-d61e0c2672a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:10.918993 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08432454-af4d-4b81-bb75-d61e0c2672a3 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:10.938741 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1053/2064] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:33:10.940139 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f8ae4-0298-42aa-9e31-cf338757c03d admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:10.952977 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f8ae4-0298-42aa-9e31-cf338757c03d admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:10.959604 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98b8bf-b88f-4336-9f86-04dcff5165b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:10.961056 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98b8bf-b88f-4336-9f86-04dcff5165b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:10.961444 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98b8bf-b88f-4336-9f86-04dcff5165b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:10.961847 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98b8bf-b88f-4336-9f86-04dcff5165b9 admin admin] PATH_INFO: `/v3/projects/979473217c604b749eb3ed2922e24939/users/75a97d18548d4ec48f07fc2bca608028/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:10.992033 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc98b8bf-b88f-4336-9f86-04dcff5165b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=979473217c604b749eb3ed2922e24939, user_id=75a97d18548d4ec48f07fc2bca608028, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:10.994931 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc98b8bf-b88f-4336-9f86-04dcff5165b9 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:11.019932 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1012/2065] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:33:10 2022] PUT /identity/v3/projects/979473217c604b749eb3ed2922e24939/users/75a97d18548d4ec48f07fc2bca608028/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:11.030427 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e310ee-0bbd-441f-b594-301741e8dd40 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:11.030717 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e310ee-0bbd-441f-b594-301741e8dd40 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:11.031002 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e310ee-0bbd-441f-b594-301741e8dd40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:11.070279 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-80e310ee-0bbd-441f-b594-301741e8dd40 None None] MFA Rules not processed for user `75a97d18548d4ec48f07fc2bca608028`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:11.071871 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-80e310ee-0bbd-441f-b594-301741e8dd40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:11.196620 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1054/2066] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:11 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:33:11.223227 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98b8bf-b88f-4336-9f86-04dcff5165b9 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:11.226034 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-dc98b8bf-b88f-4336-9f86-04dcff5165b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:11.320209 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:11.326486 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98b8bf-b88f-4336-9f86-04dcff5165b9 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:11.353269 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22ee47-dc5c-4004-97b3-41b6b5e7e904 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea874ca4088d4e6c83c9d0497323cf6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:11.354275 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22ee47-dc5c-4004-97b3-41b6b5e7e904 service glance] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:11.354588 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22ee47-dc5c-4004-97b3-41b6b5e7e904 service glance] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:11.355048 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22ee47-dc5c-4004-97b3-41b6b5e7e904 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:11.380311 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a22ee47-dc5c-4004-97b3-41b6b5e7e904 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:11.382099 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a22ee47-dc5c-4004-97b3-41b6b5e7e904 service glance] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:11.400612 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1013/2067] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:33:11 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:33:12.427641 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e310ee-0bbd-441f-b594-301741e8dd40 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:12.429815 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-80e310ee-0bbd-441f-b594-301741e8dd40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:12.508514 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e310ee-0bbd-441f-b594-301741e8dd40 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:12.522618 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9864ac-e9e5-4e16-b6dd-20461fa540ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:12.523814 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9864ac-e9e5-4e16-b6dd-20461fa540ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:12.524173 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9864ac-e9e5-4e16-b6dd-20461fa540ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:12.524608 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9864ac-e9e5-4e16-b6dd-20461fa540ea admin admin] PATH_INFO: `/v3/users/02d73e3baf9e434fabb2785ade49fd51` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:12.526041 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9864ac-e9e5-4e16-b6dd-20461fa540ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=02d73e3baf9e434fabb2785ade49fd51)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:12.528093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9864ac-e9e5-4e16-b6dd-20461fa540ea admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:12.637720 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4d9864ac-e9e5-4e16-b6dd-20461fa540ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02d73e3baf9e434fabb2785ade49fd51'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:12.659216 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4d9864ac-e9e5-4e16-b6dd-20461fa540ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02d73e3baf9e434fabb2785ade49fd51'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:12.660891 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4d9864ac-e9e5-4e16-b6dd-20461fa540ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02d73e3baf9e434fabb2785ade49fd51'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:12.661873 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4d9864ac-e9e5-4e16-b6dd-20461fa540ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02d73e3baf9e434fabb2785ade49fd51'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:12.675903 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4d9864ac-e9e5-4e16-b6dd-20461fa540ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02d73e3baf9e434fabb2785ade49fd51'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:12.697139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:12.712040 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1055/2068] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:33:12 2022] DELETE /identity/v3/users/02d73e3baf9e434fabb2785ade49fd51 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:33:12.713160 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399432ee-3d46-4380-b937-a66607a3eb0d None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:12.714122 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399432ee-3d46-4380-b937-a66607a3eb0d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:12.714580 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399432ee-3d46-4380-b937-a66607a3eb0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:12.737617 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-399432ee-3d46-4380-b937-a66607a3eb0d None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:12.739242 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-399432ee-3d46-4380-b937-a66607a3eb0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:12.904086 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1014/2069] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:12.912945 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9864ac-e9e5-4e16-b6dd-20461fa540ea admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:12.914803 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4d9864ac-e9e5-4e16-b6dd-20461fa540ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:13.054442 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9864ac-e9e5-4e16-b6dd-20461fa540ea admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:13.058907 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2f33e-a259-499a-85a0-56b290c204ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:13.060016 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2f33e-a259-499a-85a0-56b290c204ca service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:13.060477 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2f33e-a259-499a-85a0-56b290c204ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:13.060861 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2f33e-a259-499a-85a0-56b290c204ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:13.075388 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c2f33e-a259-499a-85a0-56b290c204ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:13.077133 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c2f33e-a259-499a-85a0-56b290c204ca service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:13.096813 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1056/2070] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:33:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:33:14.447951 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399432ee-3d46-4380-b937-a66607a3eb0d None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:14.450452 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-399432ee-3d46-4380-b937-a66607a3eb0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:14.477284 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399432ee-3d46-4380-b937-a66607a3eb0d None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:14.486554 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5104d5b-8cf0-460a-8aa9-060e6217539d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:14.488165 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5104d5b-8cf0-460a-8aa9-060e6217539d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:14.488672 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5104d5b-8cf0-460a-8aa9-060e6217539d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:14.489191 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5104d5b-8cf0-460a-8aa9-060e6217539d admin admin] PATH_INFO: `/v3/projects/dd923b1d88e141b4b4748fff9395b46f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:14.493452 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5104d5b-8cf0-460a-8aa9-060e6217539d admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd923b1d88e141b4b4748fff9395b46f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:14.498929 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5104d5b-8cf0-460a-8aa9-060e6217539d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:14.674202 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c5104d5b-8cf0-460a-8aa9-060e6217539d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd923b1d88e141b4b4748fff9395b46f', 'request_id': 'req-c5104d5b-8cf0-460a-8aa9-060e6217539d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:14.694632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c5104d5b-8cf0-460a-8aa9-060e6217539d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd923b1d88e141b4b4748fff9395b46f', 'request_id': 'req-c5104d5b-8cf0-460a-8aa9-060e6217539d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:14.720519 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c5104d5b-8cf0-460a-8aa9-060e6217539d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd923b1d88e141b4b4748fff9395b46f', 'request_id': 'req-c5104d5b-8cf0-460a-8aa9-060e6217539d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:14.753618 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c5104d5b-8cf0-460a-8aa9-060e6217539d admin admin] The token cache is being invalidate because project dd923b1d88e141b4b4748fff9395b46f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:33:14.754097 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c5104d5b-8cf0-460a-8aa9-060e6217539d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:14.755835 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c5104d5b-8cf0-460a-8aa9-060e6217539d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:14.759649 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1015/2071] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:33:14 2022] DELETE /identity/v3/projects/dd923b1d88e141b4b4748fff9395b46f => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:14.769558 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac18d6-aa2e-471b-ab5a-0ebcb4b1fc44 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:14.770550 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac18d6-aa2e-471b-ab5a-0ebcb4b1fc44 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:14.777504 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac18d6-aa2e-471b-ab5a-0ebcb4b1fc44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:14.848230 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-16ac18d6-aa2e-471b-ab5a-0ebcb4b1fc44 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:14.850815 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-16ac18d6-aa2e-471b-ab5a-0ebcb4b1fc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:15.046616 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1057/2072] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:15.054613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84220f6-5ea4-4948-bcbe-a3ec7e5a3d49 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:15.056091 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84220f6-5ea4-4948-bcbe-a3ec7e5a3d49 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:15.056091 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84220f6-5ea4-4948-bcbe-a3ec7e5a3d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:15.115585 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-f84220f6-5ea4-4948-bcbe-a3ec7e5a3d49 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:15.118486 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f84220f6-5ea4-4948-bcbe-a3ec7e5a3d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:15.136080 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1016/2073] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:15.140944 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac18d6-aa2e-471b-ab5a-0ebcb4b1fc44 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:15.152831 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac18d6-aa2e-471b-ab5a-0ebcb4b1fc44 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:15.159571 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dee0f9-e962-4bc0-a838-546a49f67d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:15.160337 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dee0f9-e962-4bc0-a838-546a49f67d63 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:15.160607 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dee0f9-e962-4bc0-a838-546a49f67d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:15.160883 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dee0f9-e962-4bc0-a838-546a49f67d63 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:15.161833 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dee0f9-e962-4bc0-a838-546a49f67d63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:33:15.161956 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dee0f9-e962-4bc0-a838-546a49f67d63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:15.163886 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dee0f9-e962-4bc0-a838-546a49f67d63 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:15.181679 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1058/2074] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:33:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:33:15.186053 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84220f6-5ea4-4948-bcbe-a3ec7e5a3d49 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:15.199936 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84220f6-5ea4-4948-bcbe-a3ec7e5a3d49 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:15.205878 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ffec1-baf2-4808-97d0-469609c6da07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:15.206680 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ffec1-baf2-4808-97d0-469609c6da07 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:15.207018 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ffec1-baf2-4808-97d0-469609c6da07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:15.207356 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ffec1-baf2-4808-97d0-469609c6da07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:15.208184 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070ffec1-baf2-4808-97d0-469609c6da07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:15.211563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070ffec1-baf2-4808-97d0-469609c6da07 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:15.254280 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1017/2075] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:33:15 2022] POST /identity/v3/projects => generated 373 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:33:15.259073 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dee0f9-e962-4bc0-a838-546a49f67d63 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:15.279440 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dee0f9-e962-4bc0-a838-546a49f67d63 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:15.363059 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89be011-374e-4c4f-ba6c-e989028a3f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:15.363759 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89be011-374e-4c4f-ba6c-e989028a3f6b admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:15.364044 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89be011-374e-4c4f-ba6c-e989028a3f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:15.364319 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89be011-374e-4c4f-ba6c-e989028a3f6b admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:15.365024 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89be011-374e-4c4f-ba6c-e989028a3f6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:15.367111 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89be011-374e-4c4f-ba6c-e989028a3f6b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:15.416497 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1059/2076] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:33:15 2022] POST /identity/v3/users => generated 287 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:33:15.422853 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ffec1-baf2-4808-97d0-469609c6da07 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:15.444722 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ffec1-baf2-4808-97d0-469609c6da07 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:15.451525 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f3805-2339-48db-8542-6fde2ab66b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:15.452273 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f3805-2339-48db-8542-6fde2ab66b9d admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:15.452801 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f3805-2339-48db-8542-6fde2ab66b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:15.453133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f3805-2339-48db-8542-6fde2ab66b9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:15.454064 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231f3805-2339-48db-8542-6fde2ab66b9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:15.457474 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231f3805-2339-48db-8542-6fde2ab66b9d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:15.473448 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1018/2077] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:15 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:33:15.478013 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89be011-374e-4c4f-ba6c-e989028a3f6b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:15.491109 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89be011-374e-4c4f-ba6c-e989028a3f6b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:15.496705 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02038d48-e934-4916-9485-5e2dad4bb956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:15.497344 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02038d48-e934-4916-9485-5e2dad4bb956 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:15.497642 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02038d48-e934-4916-9485-5e2dad4bb956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:15.500870 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02038d48-e934-4916-9485-5e2dad4bb956 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:15.501771 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02038d48-e934-4916-9485-5e2dad4bb956 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:15.503680 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02038d48-e934-4916-9485-5e2dad4bb956 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:15.515682 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1060/2078] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 21:33:15.520846 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f3805-2339-48db-8542-6fde2ab66b9d admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:15.538692 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f3805-2339-48db-8542-6fde2ab66b9d admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:15.546980 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa027b0-725a-463f-9196-2108ae1d0605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:15.548642 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa027b0-725a-463f-9196-2108ae1d0605 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:15.549238 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa027b0-725a-463f-9196-2108ae1d0605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:15.549670 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa027b0-725a-463f-9196-2108ae1d0605 admin admin] PATH_INFO: `/v3/projects/f9c4f4baaf8f4a258171bf97abdc8905/users/47575a76d62b4a109fda03f4bf016fc6/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:15.570594 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa027b0-725a-463f-9196-2108ae1d0605 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9c4f4baaf8f4a258171bf97abdc8905, user_id=47575a76d62b4a109fda03f4bf016fc6, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:15.573651 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa027b0-725a-463f-9196-2108ae1d0605 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:15.591967 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1019/2079] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:33:15 2022] PUT /identity/v3/projects/f9c4f4baaf8f4a258171bf97abdc8905/users/47575a76d62b4a109fda03f4bf016fc6/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:15.599251 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f271a4-4cfc-45b4-b05f-43eb8ae3e339 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:15.599827 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f271a4-4cfc-45b4-b05f-43eb8ae3e339 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:15.600190 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f271a4-4cfc-45b4-b05f-43eb8ae3e339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:15.621997 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-04f271a4-4cfc-45b4-b05f-43eb8ae3e339 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:15.623416 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-04f271a4-4cfc-45b4-b05f-43eb8ae3e339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:15.628669 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:15.732476 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1061/2080] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:15.743258 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa027b0-725a-463f-9196-2108ae1d0605 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:15.745836 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0fa027b0-725a-463f-9196-2108ae1d0605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:15.795531 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:15.873793 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa027b0-725a-463f-9196-2108ae1d0605 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:15.879077 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c46205-6ce1-4155-8a1a-71365e05d752 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:15.880445 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c46205-6ce1-4155-8a1a-71365e05d752 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:15.880785 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c46205-6ce1-4155-8a1a-71365e05d752 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:15.881218 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c46205-6ce1-4155-8a1a-71365e05d752 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:15.896334 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c46205-6ce1-4155-8a1a-71365e05d752 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:15.897814 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c46205-6ce1-4155-8a1a-71365e05d752 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:15.910391 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1020/2081] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:33:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:33:17.960137 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac56ae3-9f8a-43e9-a296-6b675cf9d07a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:17.960964 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac56ae3-9f8a-43e9-a296-6b675cf9d07a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:17.961077 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac56ae3-9f8a-43e9-a296-6b675cf9d07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:18.104301 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-fac56ae3-9f8a-43e9-a296-6b675cf9d07a None None] MFA Rules not processed for user `47575a76d62b4a109fda03f4bf016fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:18.107556 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fac56ae3-9f8a-43e9-a296-6b675cf9d07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:18.280581 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1062/2082] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:18.312104 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c46205-6ce1-4155-8a1a-71365e05d752 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:18.314962 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-73c46205-6ce1-4155-8a1a-71365e05d752 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:18.453112 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c46205-6ce1-4155-8a1a-71365e05d752 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:18.460414 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e033bf-9504-4273-af6d-c60330c8930f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:18.462651 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e033bf-9504-4273-af6d-c60330c8930f service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:18.462994 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e033bf-9504-4273-af6d-c60330c8930f service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:18.463336 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e033bf-9504-4273-af6d-c60330c8930f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:18.494747 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e033bf-9504-4273-af6d-c60330c8930f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:18.510931 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e033bf-9504-4273-af6d-c60330c8930f service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:18.539608 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1021/2083] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:33:18 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:33:19.375032 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac56ae3-9f8a-43e9-a296-6b675cf9d07a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:19.378438 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fac56ae3-9f8a-43e9-a296-6b675cf9d07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:19.406989 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac56ae3-9f8a-43e9-a296-6b675cf9d07a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:19.416606 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff00811-9ba1-41d4-8d0c-706c7315f5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:19.418152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff00811-9ba1-41d4-8d0c-706c7315f5b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:19.418683 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff00811-9ba1-41d4-8d0c-706c7315f5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:19.419166 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff00811-9ba1-41d4-8d0c-706c7315f5b0 admin admin] PATH_INFO: `/v3/users/bc570204a2bb4889889a67850dd1d9fe` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:19.421573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff00811-9ba1-41d4-8d0c-706c7315f5b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc570204a2bb4889889a67850dd1d9fe)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:19.425598 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff00811-9ba1-41d4-8d0c-706c7315f5b0 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:19.516328 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cff00811-9ba1-41d4-8d0c-706c7315f5b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc570204a2bb4889889a67850dd1d9fe'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:19.540944 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cff00811-9ba1-41d4-8d0c-706c7315f5b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc570204a2bb4889889a67850dd1d9fe'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:19.541906 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cff00811-9ba1-41d4-8d0c-706c7315f5b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc570204a2bb4889889a67850dd1d9fe'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:19.542567 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cff00811-9ba1-41d4-8d0c-706c7315f5b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc570204a2bb4889889a67850dd1d9fe'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:19.552121 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cff00811-9ba1-41d4-8d0c-706c7315f5b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc570204a2bb4889889a67850dd1d9fe'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:19.577232 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1063/2084] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:33:19 2022] DELETE /identity/v3/users/bc570204a2bb4889889a67850dd1d9fe => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:19.889896 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e033bf-9504-4273-af6d-c60330c8930f service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:19.905590 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e033bf-9504-4273-af6d-c60330c8930f service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:20.015730 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f257c1eb-d715-4e68-883b-265bc909d4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:20.016599 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257c1eb-d715-4e68-883b-265bc909d4ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:20.016878 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257c1eb-d715-4e68-883b-265bc909d4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:20.017189 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257c1eb-d715-4e68-883b-265bc909d4ac admin admin] PATH_INFO: `/v3/projects/6166dc89ce0545c1bbb8adb947034e79` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:20.019106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f257c1eb-d715-4e68-883b-265bc909d4ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=6166dc89ce0545c1bbb8adb947034e79)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:20.021163 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f257c1eb-d715-4e68-883b-265bc909d4ac admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:20.105447 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-f257c1eb-d715-4e68-883b-265bc909d4ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6166dc89ce0545c1bbb8adb947034e79', 'request_id': 'req-f257c1eb-d715-4e68-883b-265bc909d4ac'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:20.115233 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-f257c1eb-d715-4e68-883b-265bc909d4ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6166dc89ce0545c1bbb8adb947034e79', 'request_id': 'req-f257c1eb-d715-4e68-883b-265bc909d4ac'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:20.122304 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:20.131267 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-f257c1eb-d715-4e68-883b-265bc909d4ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6166dc89ce0545c1bbb8adb947034e79', 'request_id': 'req-f257c1eb-d715-4e68-883b-265bc909d4ac'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:20.151596 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-f257c1eb-d715-4e68-883b-265bc909d4ac admin admin] The token cache is being invalidate because project 6166dc89ce0545c1bbb8adb947034e79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:33:20.152009 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-f257c1eb-d715-4e68-883b-265bc909d4ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:20.152904 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-f257c1eb-d715-4e68-883b-265bc909d4ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:20.154897 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1022/2085] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:33:19 2022] DELETE /identity/v3/projects/6166dc89ce0545c1bbb8adb947034e79 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:20.162085 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edafb42-5c0c-44e7-b27c-a4b2aef65035 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:20.162944 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edafb42-5c0c-44e7-b27c-a4b2aef65035 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:20.163484 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edafb42-5c0c-44e7-b27c-a4b2aef65035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:20.191077 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-4edafb42-5c0c-44e7-b27c-a4b2aef65035 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:20.193366 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4edafb42-5c0c-44e7-b27c-a4b2aef65035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:20.202946 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:20.302329 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1064/2086] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:20.313002 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78979504-c69c-4e45-8e02-630a35d56fb7 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:20.313493 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78979504-c69c-4e45-8e02-630a35d56fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:20.313982 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78979504-c69c-4e45-8e02-630a35d56fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:20.349291 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-78979504-c69c-4e45-8e02-630a35d56fb7 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:20.351457 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-78979504-c69c-4e45-8e02-630a35d56fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:20.366849 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1023/2087] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:20.371545 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edafb42-5c0c-44e7-b27c-a4b2aef65035 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:20.386384 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edafb42-5c0c-44e7-b27c-a4b2aef65035 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:20.397162 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e741c88-3ef1-4297-ae8e-7d37b870e9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:20.398429 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e741c88-3ef1-4297-ae8e-7d37b870e9c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:20.400256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e741c88-3ef1-4297-ae8e-7d37b870e9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:20.400878 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e741c88-3ef1-4297-ae8e-7d37b870e9c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:20.402411 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e741c88-3ef1-4297-ae8e-7d37b870e9c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:33:20.403060 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e741c88-3ef1-4297-ae8e-7d37b870e9c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:20.406872 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e741c88-3ef1-4297-ae8e-7d37b870e9c7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:20.429289 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1065/2088] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:33:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:33:20.436288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78979504-c69c-4e45-8e02-630a35d56fb7 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:20.452877 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78979504-c69c-4e45-8e02-630a35d56fb7 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:20.459767 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fdeee1-600e-4561-98e4-edea6497fa9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:20.460586 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fdeee1-600e-4561-98e4-edea6497fa9c admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:20.460961 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fdeee1-600e-4561-98e4-edea6497fa9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:20.461381 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fdeee1-600e-4561-98e4-edea6497fa9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:20.462430 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fdeee1-600e-4561-98e4-edea6497fa9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:20.464633 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fdeee1-600e-4561-98e4-edea6497fa9c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:20.524048 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1024/2089] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:33:20 2022] POST /identity/v3/projects => generated 379 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:33:20.529092 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e741c88-3ef1-4297-ae8e-7d37b870e9c7 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:20.541176 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e741c88-3ef1-4297-ae8e-7d37b870e9c7 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:20.643705 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86910e15-498f-4028-b3a0-49c402c3df2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:20.644869 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86910e15-498f-4028-b3a0-49c402c3df2a admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:20.645400 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86910e15-498f-4028-b3a0-49c402c3df2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:20.645888 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86910e15-498f-4028-b3a0-49c402c3df2a admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:20.647220 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86910e15-498f-4028-b3a0-49c402c3df2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:20.651023 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86910e15-498f-4028-b3a0-49c402c3df2a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:20.683504 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1066/2090] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:33:20 2022] POST /identity/v3/users => generated 290 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:33:20.688419 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fdeee1-600e-4561-98e4-edea6497fa9c admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:20.701199 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fdeee1-600e-4561-98e4-edea6497fa9c admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:20.707895 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f05d70-f80d-44a7-84d8-ec02bfb1db67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:20.708602 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f05d70-f80d-44a7-84d8-ec02bfb1db67 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:20.709006 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f05d70-f80d-44a7-84d8-ec02bfb1db67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:20.709343 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f05d70-f80d-44a7-84d8-ec02bfb1db67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:20.710362 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f05d70-f80d-44a7-84d8-ec02bfb1db67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:20.712228 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f05d70-f80d-44a7-84d8-ec02bfb1db67 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:20.732100 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1025/2091] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:33:20.734152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86910e15-498f-4028-b3a0-49c402c3df2a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:20.748570 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86910e15-498f-4028-b3a0-49c402c3df2a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:20.754185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb748f-bc11-4e31-99eb-83d6fc85089c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:20.754926 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb748f-bc11-4e31-99eb-83d6fc85089c admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:20.755212 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb748f-bc11-4e31-99eb-83d6fc85089c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:20.755518 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb748f-bc11-4e31-99eb-83d6fc85089c admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:20.756743 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cb748f-bc11-4e31-99eb-83d6fc85089c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:20.759030 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cb748f-bc11-4e31-99eb-83d6fc85089c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:20.777667 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1067/2092] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:33:20.784375 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f05d70-f80d-44a7-84d8-ec02bfb1db67 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:20.800637 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f05d70-f80d-44a7-84d8-ec02bfb1db67 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:20.807352 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a9f7d-cb36-40f7-9669-1be50ab08ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:20.808779 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a9f7d-cb36-40f7-9669-1be50ab08ad7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:20.809089 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a9f7d-cb36-40f7-9669-1be50ab08ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:20.809430 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a9f7d-cb36-40f7-9669-1be50ab08ad7 admin admin] PATH_INFO: `/v3/projects/4b01a800e25c4d0c8f6391b96317064b/users/7ce81e1b13104352a1b67612a8de957b/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:20.837901 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2a9f7d-cb36-40f7-9669-1be50ab08ad7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b01a800e25c4d0c8f6391b96317064b, user_id=7ce81e1b13104352a1b67612a8de957b, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:20.841047 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2a9f7d-cb36-40f7-9669-1be50ab08ad7 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:20.894339 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1026/2093] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:33:20 2022] PUT /identity/v3/projects/4b01a800e25c4d0c8f6391b96317064b/users/7ce81e1b13104352a1b67612a8de957b/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:20.909066 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb3edd-fa2f-406a-a867-1e6b0df7f487 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:20.909654 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb3edd-fa2f-406a-a867-1e6b0df7f487 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:20.909944 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb3edd-fa2f-406a-a867-1e6b0df7f487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:20.945641 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b7fb3edd-fa2f-406a-a867-1e6b0df7f487 None None] MFA Rules not processed for user `7ce81e1b13104352a1b67612a8de957b`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:20.946913 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b7fb3edd-fa2f-406a-a867-1e6b0df7f487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:21.034917 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1068/2094] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:21.040205 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a9f7d-cb36-40f7-9669-1be50ab08ad7 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:21.166656 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a9f7d-cb36-40f7-9669-1be50ab08ad7 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:21.187738 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e0e3d7-de3b-4955-8b37-e488b2fafe3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:21.189355 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0e3d7-de3b-4955-8b37-e488b2fafe3a admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:21.189893 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0e3d7-de3b-4955-8b37-e488b2fafe3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:21.190687 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0e3d7-de3b-4955-8b37-e488b2fafe3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:21.192194 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e0e3d7-de3b-4955-8b37-e488b2fafe3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:21.197170 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e0e3d7-de3b-4955-8b37-e488b2fafe3a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:21.244286 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ccb78-73d1-4542-b2b0-0bcc7f9810a9 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:21.244791 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ccb78-73d1-4542-b2b0-0bcc7f9810a9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:21.245303 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ccb78-73d1-4542-b2b0-0bcc7f9810a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:21.255789 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1027/2095] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:33:21 2022] POST /identity/v3/projects => generated 379 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:33:21.257181 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e0e3d7-de3b-4955-8b37-e488b2fafe3a admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:21.280642 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:21.299286 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-683ccb78-73d1-4542-b2b0-0bcc7f9810a9 None None] MFA Rules not processed for user `47575a76d62b4a109fda03f4bf016fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:21.304321 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-683ccb78-73d1-4542-b2b0-0bcc7f9810a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:21.399958 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e0e3d7-de3b-4955-8b37-e488b2fafe3a admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:21.406410 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7e504-1f37-4512-99f0-2311d7b01d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:21.407520 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7e504-1f37-4512-99f0-2311d7b01d83 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:21.408029 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7e504-1f37-4512-99f0-2311d7b01d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:21.408451 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7e504-1f37-4512-99f0-2311d7b01d83 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:21.409890 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a7e504-1f37-4512-99f0-2311d7b01d83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:21.413077 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a7e504-1f37-4512-99f0-2311d7b01d83 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:21.435942 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:DELETE=2,keystone:INSERT=6 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:21.438780 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1069/2096] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:33:21.450463 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1028/2097] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:33:21 2022] POST /identity/v3/users => generated 296 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:33:21.452574 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ccb78-73d1-4542-b2b0-0bcc7f9810a9 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:21.469895 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ccb78-73d1-4542-b2b0-0bcc7f9810a9 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:21.476335 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcebaa2-7525-4ec4-b991-078f06f53dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:21.477208 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcebaa2-7525-4ec4-b991-078f06f53dcb admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:21.477564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcebaa2-7525-4ec4-b991-078f06f53dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:21.478023 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcebaa2-7525-4ec4-b991-078f06f53dcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:21.478949 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcebaa2-7525-4ec4-b991-078f06f53dcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:21.481712 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcebaa2-7525-4ec4-b991-078f06f53dcb admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:21.500153 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1070/2098] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:33:21.507306 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7e504-1f37-4512-99f0-2311d7b01d83 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:21.524995 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7e504-1f37-4512-99f0-2311d7b01d83 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:21.535247 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192e66f-feb5-4335-977d-5c2e11be9e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:21.536219 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192e66f-feb5-4335-977d-5c2e11be9e68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:21.536498 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192e66f-feb5-4335-977d-5c2e11be9e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:21.536771 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192e66f-feb5-4335-977d-5c2e11be9e68 admin admin] PATH_INFO: `/v3/projects/c87d5ce6ca6a4299aeba8d22c11eb44a/users/b2f8d40a3f9347b999b485753af6ded6/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:21.557771 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6192e66f-feb5-4335-977d-5c2e11be9e68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c87d5ce6ca6a4299aeba8d22c11eb44a, user_id=b2f8d40a3f9347b999b485753af6ded6, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:21.560557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6192e66f-feb5-4335-977d-5c2e11be9e68 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:21.578850 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1029/2099] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:33:21 2022] PUT /identity/v3/projects/c87d5ce6ca6a4299aeba8d22c11eb44a/users/b2f8d40a3f9347b999b485753af6ded6/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:21.584063 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcebaa2-7525-4ec4-b991-078f06f53dcb admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:21.595636 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcebaa2-7525-4ec4-b991-078f06f53dcb admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:21.686633 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c9c81-9700-4986-9610-f75261f977ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:21.688058 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c9c81-9700-4986-9610-f75261f977ca admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:21.688519 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c9c81-9700-4986-9610-f75261f977ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:21.689385 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c9c81-9700-4986-9610-f75261f977ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:21.692168 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c9c81-9700-4986-9610-f75261f977ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:21.694732 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c9c81-9700-4986-9610-f75261f977ca admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:21.711828 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1071/2100] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 21:33:21.718717 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192e66f-feb5-4335-977d-5c2e11be9e68 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:21.745102 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192e66f-feb5-4335-977d-5c2e11be9e68 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:21.754794 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af568869-885d-40b2-806a-6fd1f9c15d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:21.756462 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af568869-885d-40b2-806a-6fd1f9c15d43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:21.756950 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af568869-885d-40b2-806a-6fd1f9c15d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:21.760575 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af568869-885d-40b2-806a-6fd1f9c15d43 admin admin] PATH_INFO: `/v3/projects/c87d5ce6ca6a4299aeba8d22c11eb44a/users/b2f8d40a3f9347b999b485753af6ded6/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:21.761817 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af568869-885d-40b2-806a-6fd1f9c15d43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c87d5ce6ca6a4299aeba8d22c11eb44a, user_id=b2f8d40a3f9347b999b485753af6ded6, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:21.769668 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af568869-885d-40b2-806a-6fd1f9c15d43 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:21.794343 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1030/2101] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:33:21 2022] PUT /identity/v3/projects/c87d5ce6ca6a4299aeba8d22c11eb44a/users/b2f8d40a3f9347b999b485753af6ded6/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:21.811599 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b166d9-db59-43f9-9d42-967c138b03c1 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:21.812095 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b166d9-db59-43f9-9d42-967c138b03c1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:21.812580 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b166d9-db59-43f9-9d42-967c138b03c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:21.885044 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-78b166d9-db59-43f9-9d42-967c138b03c1 None None] MFA Rules not processed for user `b2f8d40a3f9347b999b485753af6ded6`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:21.887406 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-78b166d9-db59-43f9-9d42-967c138b03c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:22.037042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1072/2102] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:21 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:22.057483 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af568869-885d-40b2-806a-6fd1f9c15d43 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:22.064704 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-af568869-885d-40b2-806a-6fd1f9c15d43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:22.171180 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af568869-885d-40b2-806a-6fd1f9c15d43 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:22.178811 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895903af-973a-4d00-b5f0-7e79e0fe4c3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:22.179924 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895903af-973a-4d00-b5f0-7e79e0fe4c3f service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:22.180391 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895903af-973a-4d00-b5f0-7e79e0fe4c3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:22.181907 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895903af-973a-4d00-b5f0-7e79e0fe4c3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:22.199920 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895903af-973a-4d00-b5f0-7e79e0fe4c3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:22.203067 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895903af-973a-4d00-b5f0-7e79e0fe4c3f service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:22.225106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1031/2103] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:33:22 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:33:22.348339 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b166d9-db59-43f9-9d42-967c138b03c1 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:22.371310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b166d9-db59-43f9-9d42-967c138b03c1 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:22.380459 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd6e09-0afd-477c-9692-69cb59c39825 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:22.381298 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd6e09-0afd-477c-9692-69cb59c39825 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:22.381652 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd6e09-0afd-477c-9692-69cb59c39825 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:22.381959 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd6e09-0afd-477c-9692-69cb59c39825 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:22.414105 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:22.473497 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fd6e09-0afd-477c-9692-69cb59c39825 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:22.478547 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fd6e09-0afd-477c-9692-69cb59c39825 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:22.498845 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1073/2104] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:33:22 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:33:32.214533 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=42,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:32.489917 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=18 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:38.134766 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895903af-973a-4d00-b5f0-7e79e0fe4c3f service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:38.137052 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-895903af-973a-4d00-b5f0-7e79e0fe4c3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:38.257732 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895903af-973a-4d00-b5f0-7e79e0fe4c3f service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:38.266213 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aecb2e-a834-412e-9388-7d8b2d75190d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:38.267386 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aecb2e-a834-412e-9388-7d8b2d75190d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:38.267850 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aecb2e-a834-412e-9388-7d8b2d75190d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:38.268301 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aecb2e-a834-412e-9388-7d8b2d75190d admin admin] PATH_INFO: `/v3/users/8404904966c84007a5f39ca6f58b10dd` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:38.269870 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aecb2e-a834-412e-9388-7d8b2d75190d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8404904966c84007a5f39ca6f58b10dd)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:38.272391 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aecb2e-a834-412e-9388-7d8b2d75190d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:38.354874 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd6e09-0afd-477c-9692-69cb59c39825 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:38.363538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-36fd6e09-0afd-477c-9692-69cb59c39825 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:38.392971 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd6e09-0afd-477c-9692-69cb59c39825 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:38.407924 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9244216-cbbe-4bef-a126-5a92d865cd52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:38.409233 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9244216-cbbe-4bef-a126-5a92d865cd52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:38.409717 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9244216-cbbe-4bef-a126-5a92d865cd52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:38.410197 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9244216-cbbe-4bef-a126-5a92d865cd52 admin admin] PATH_INFO: `/v3/users/9a04801eafc546058ca61be090668788` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:38.412597 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9244216-cbbe-4bef-a126-5a92d865cd52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a04801eafc546058ca61be090668788)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:38.414480 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-50aecb2e-a834-412e-9388-7d8b2d75190d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8404904966c84007a5f39ca6f58b10dd'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:38.416370 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-50aecb2e-a834-412e-9388-7d8b2d75190d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8404904966c84007a5f39ca6f58b10dd'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:38.416778 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9244216-cbbe-4bef-a126-5a92d865cd52 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:38.443205 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-50aecb2e-a834-412e-9388-7d8b2d75190d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8404904966c84007a5f39ca6f58b10dd'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:38.452807 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-50aecb2e-a834-412e-9388-7d8b2d75190d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8404904966c84007a5f39ca6f58b10dd'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:38.464954 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-50aecb2e-a834-412e-9388-7d8b2d75190d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8404904966c84007a5f39ca6f58b10dd'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:38.479164 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1032/2105] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:33:38 2022] DELETE /identity/v3/users/8404904966c84007a5f39ca6f58b10dd => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:38.486191 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c906c43-74f9-4872-9867-7fc6dd2538e4 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:38.487723 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c906c43-74f9-4872-9867-7fc6dd2538e4 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:38.488229 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c906c43-74f9-4872-9867-7fc6dd2538e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:38.533215 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-4c906c43-74f9-4872-9867-7fc6dd2538e4 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:38.534734 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e9244216-cbbe-4bef-a126-5a92d865cd52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a04801eafc546058ca61be090668788'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:38.535405 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4c906c43-74f9-4872-9867-7fc6dd2538e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:38.567611 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e9244216-cbbe-4bef-a126-5a92d865cd52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a04801eafc546058ca61be090668788'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:38.569242 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e9244216-cbbe-4bef-a126-5a92d865cd52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a04801eafc546058ca61be090668788'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:38.570303 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e9244216-cbbe-4bef-a126-5a92d865cd52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a04801eafc546058ca61be090668788'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:38.582322 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e9244216-cbbe-4bef-a126-5a92d865cd52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a04801eafc546058ca61be090668788'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:38.600975 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1074/2106] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:33:38 2022] DELETE /identity/v3/users/9a04801eafc546058ca61be090668788 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:38.606568 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91b82d-ee52-4230-8f4b-10b282c43676 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:38.607012 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91b82d-ee52-4230-8f4b-10b282c43676 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:38.607355 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91b82d-ee52-4230-8f4b-10b282c43676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:38.631512 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-7d91b82d-ee52-4230-8f4b-10b282c43676 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:38.632823 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7d91b82d-ee52-4230-8f4b-10b282c43676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:38.669964 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1033/2107] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:33:38.691000 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c906c43-74f9-4872-9867-7fc6dd2538e4 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:38.696481 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4c906c43-74f9-4872-9867-7fc6dd2538e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:38.755461 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:38.800578 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1075/2108] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:38.819872 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d91b82d-ee52-4230-8f4b-10b282c43676 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:38.837282 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c906c43-74f9-4872-9867-7fc6dd2538e4 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:38.926195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d91b82d-ee52-4230-8f4b-10b282c43676 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:38.931430 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d032d270-ea8f-43e7-bb6f-f79ca0f199f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:38.932121 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032d270-ea8f-43e7-bb6f-f79ca0f199f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:38.932457 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032d270-ea8f-43e7-bb6f-f79ca0f199f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:38.932739 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032d270-ea8f-43e7-bb6f-f79ca0f199f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:38.946596 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d032d270-ea8f-43e7-bb6f-f79ca0f199f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:38.948170 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d032d270-ea8f-43e7-bb6f-f79ca0f199f5 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:38.950970 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6331201-0176-4ed7-9cbe-20a977697d6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:38.951723 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6331201-0176-4ed7-9cbe-20a977697d6e service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:38.952012 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6331201-0176-4ed7-9cbe-20a977697d6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:38.952632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6331201-0176-4ed7-9cbe-20a977697d6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:38.962504 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1076/2109] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:33:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:33:38.963198 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d032d270-ea8f-43e7-bb6f-f79ca0f199f5 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:38.964720 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d032d270-ea8f-43e7-bb6f-f79ca0f199f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:38.967748 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6331201-0176-4ed7-9cbe-20a977697d6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:38.971278 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6331201-0176-4ed7-9cbe-20a977697d6e service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:38.980438 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d032d270-ea8f-43e7-bb6f-f79ca0f199f5 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:38.988444 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1034/2110] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:33:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:33:38.989988 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b99c5b-93c2-47d2-b9a9-9c9fc652adbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:38.991454 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b99c5b-93c2-47d2-b9a9-9c9fc652adbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:38.991856 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b99c5b-93c2-47d2-b9a9-9c9fc652adbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:38.992348 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b99c5b-93c2-47d2-b9a9-9c9fc652adbb admin admin] PATH_INFO: `/v3/users/7ce81e1b13104352a1b67612a8de957b` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:38.994040 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b99c5b-93c2-47d2-b9a9-9c9fc652adbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ce81e1b13104352a1b67612a8de957b)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:38.997496 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b99c5b-93c2-47d2-b9a9-9c9fc652adbb admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:39.021084 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:39.102488 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-20b99c5b-93c2-47d2-b9a9-9c9fc652adbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ce81e1b13104352a1b67612a8de957b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:39.131348 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-20b99c5b-93c2-47d2-b9a9-9c9fc652adbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ce81e1b13104352a1b67612a8de957b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:39.132127 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-20b99c5b-93c2-47d2-b9a9-9c9fc652adbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ce81e1b13104352a1b67612a8de957b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:39.139632 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-20b99c5b-93c2-47d2-b9a9-9c9fc652adbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ce81e1b13104352a1b67612a8de957b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:39.150271 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-20b99c5b-93c2-47d2-b9a9-9c9fc652adbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ce81e1b13104352a1b67612a8de957b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:39.174225 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1077/2111] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:33:38 2022] DELETE /identity/v3/users/7ce81e1b13104352a1b67612a8de957b => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:39.182334 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec95504-adc0-4c2b-90e9-a30b56f67b20 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:39.182925 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec95504-adc0-4c2b-90e9-a30b56f67b20 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:39.183335 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec95504-adc0-4c2b-90e9-a30b56f67b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:39.217236 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-eec95504-adc0-4c2b-90e9-a30b56f67b20 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:39.221120 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-eec95504-adc0-4c2b-90e9-a30b56f67b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:39.419050 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1035/2112] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:39.446208 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b99c5b-93c2-47d2-b9a9-9c9fc652adbb admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:39.448215 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-20b99c5b-93c2-47d2-b9a9-9c9fc652adbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:39.499165 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec95504-adc0-4c2b-90e9-a30b56f67b20 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:39.502766 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-eec95504-adc0-4c2b-90e9-a30b56f67b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:39.539662 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec95504-adc0-4c2b-90e9-a30b56f67b20 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:39.578266 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8118bcc4-b6f1-4011-8925-91a3ea8d184a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:39.578598 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b99c5b-93c2-47d2-b9a9-9c9fc652adbb admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:39.584132 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8118bcc4-b6f1-4011-8925-91a3ea8d184a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:39.584659 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8118bcc4-b6f1-4011-8925-91a3ea8d184a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:39.585150 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8118bcc4-b6f1-4011-8925-91a3ea8d184a admin admin] PATH_INFO: `/v3/projects/6be4490715a14e6ba6205157a0e28743` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:39.585974 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0db4bad-5313-45a0-884f-bbfc42775890 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:39.587095 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0db4bad-5313-45a0-884f-bbfc42775890 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:39.587601 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0db4bad-5313-45a0-884f-bbfc42775890 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:39.588449 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0db4bad-5313-45a0-884f-bbfc42775890 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:39.598495 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8118bcc4-b6f1-4011-8925-91a3ea8d184a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6be4490715a14e6ba6205157a0e28743)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:39.603816 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8118bcc4-b6f1-4011-8925-91a3ea8d184a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:39.609361 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0db4bad-5313-45a0-884f-bbfc42775890 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:39.613454 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0db4bad-5313-45a0-884f-bbfc42775890 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:39.641942 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1078/2113] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:33:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:33:39.644125 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0db4bad-5313-45a0-884f-bbfc42775890 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:39.646396 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a0db4bad-5313-45a0-884f-bbfc42775890 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:39.663819 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0db4bad-5313-45a0-884f-bbfc42775890 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:39.670562 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a741238-080a-47da-839d-9ad89b090dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:39.671770 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a741238-080a-47da-839d-9ad89b090dc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:39.672050 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a741238-080a-47da-839d-9ad89b090dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:39.672327 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a741238-080a-47da-839d-9ad89b090dc5 admin admin] PATH_INFO: `/v3/projects/05c47588614d40cca446f0346af9543f` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:39.674185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a741238-080a-47da-839d-9ad89b090dc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05c47588614d40cca446f0346af9543f)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:39.677231 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a741238-080a-47da-839d-9ad89b090dc5 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:39.745729 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8118bcc4-b6f1-4011-8925-91a3ea8d184a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6be4490715a14e6ba6205157a0e28743', 'request_id': 'req-8118bcc4-b6f1-4011-8925-91a3ea8d184a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:39.757190 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8118bcc4-b6f1-4011-8925-91a3ea8d184a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6be4490715a14e6ba6205157a0e28743', 'request_id': 'req-8118bcc4-b6f1-4011-8925-91a3ea8d184a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:39.776442 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8118bcc4-b6f1-4011-8925-91a3ea8d184a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6be4490715a14e6ba6205157a0e28743', 'request_id': 'req-8118bcc4-b6f1-4011-8925-91a3ea8d184a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:39.790784 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8118bcc4-b6f1-4011-8925-91a3ea8d184a admin admin] The token cache is being invalidate because project 6be4490715a14e6ba6205157a0e28743 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:33:39.791316 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8118bcc4-b6f1-4011-8925-91a3ea8d184a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:39.792387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8118bcc4-b6f1-4011-8925-91a3ea8d184a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:39.794857 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1036/2114] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:33:39 2022] DELETE /identity/v3/projects/6be4490715a14e6ba6205157a0e28743 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:39.805024 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda375c7-58da-46ff-8b5c-4389f1bd7627 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:39.805566 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda375c7-58da-46ff-8b5c-4389f1bd7627 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:39.805980 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda375c7-58da-46ff-8b5c-4389f1bd7627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:39.811598 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6a741238-080a-47da-839d-9ad89b090dc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05c47588614d40cca446f0346af9543f', 'request_id': 'req-6a741238-080a-47da-839d-9ad89b090dc5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:39.817179 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:39.823772 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6a741238-080a-47da-839d-9ad89b090dc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05c47588614d40cca446f0346af9543f', 'request_id': 'req-6a741238-080a-47da-839d-9ad89b090dc5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:39.837711 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-bda375c7-58da-46ff-8b5c-4389f1bd7627 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:39.839306 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-bda375c7-58da-46ff-8b5c-4389f1bd7627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:39.850306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6a741238-080a-47da-839d-9ad89b090dc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05c47588614d40cca446f0346af9543f', 'request_id': 'req-6a741238-080a-47da-839d-9ad89b090dc5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:39.875762 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6a741238-080a-47da-839d-9ad89b090dc5 admin admin] The token cache is being invalidate because project 05c47588614d40cca446f0346af9543f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:33:39.876446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6a741238-080a-47da-839d-9ad89b090dc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:39.877222 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6a741238-080a-47da-839d-9ad89b090dc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:39.879222 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1079/2115] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:33:39 2022] DELETE /identity/v3/projects/05c47588614d40cca446f0346af9543f => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:39.883381 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a741238-080a-47da-839d-9ad89b090dc5 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:39.885126 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6a741238-080a-47da-839d-9ad89b090dc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:39.902676 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:39.969423 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1037/2116] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:39.979522 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f91ff-1594-4fde-9b7a-8e29ee20fd4e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:39.980415 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f91ff-1594-4fde-9b7a-8e29ee20fd4e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:39.980955 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a741238-080a-47da-839d-9ad89b090dc5 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:39.981080 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f91ff-1594-4fde-9b7a-8e29ee20fd4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:39.993406 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7339733-87b6-4590-b45c-c98f782b24b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:39.993406 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7339733-87b6-4590-b45c-c98f782b24b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:39.993968 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7339733-87b6-4590-b45c-c98f782b24b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:39.994155 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7339733-87b6-4590-b45c-c98f782b24b7 admin admin] PATH_INFO: `/v3/users/857d05e10de645368a4cfe5af49c13cc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:39.996311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7339733-87b6-4590-b45c-c98f782b24b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=857d05e10de645368a4cfe5af49c13cc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:40.002406 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7339733-87b6-4590-b45c-c98f782b24b7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:40.031899 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-a35f91ff-1594-4fde-9b7a-8e29ee20fd4e None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:40.034858 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a35f91ff-1594-4fde-9b7a-8e29ee20fd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:40.052993 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1038/2117] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:40.058750 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f91ff-1594-4fde-9b7a-8e29ee20fd4e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:40.070337 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f91ff-1594-4fde-9b7a-8e29ee20fd4e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:40.075742 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d428f9-c938-4810-a849-dc00de8c8f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:40.076451 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d428f9-c938-4810-a849-dc00de8c8f15 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:40.076746 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d428f9-c938-4810-a849-dc00de8c8f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:40.077017 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d428f9-c938-4810-a849-dc00de8c8f15 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:40.077947 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d428f9-c938-4810-a849-dc00de8c8f15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:33:40.078272 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d428f9-c938-4810-a849-dc00de8c8f15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:40.080178 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d428f9-c938-4810-a849-dc00de8c8f15 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:40.095760 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1039/2118] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:33:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:33:40.097643 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c7339733-87b6-4590-b45c-c98f782b24b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '857d05e10de645368a4cfe5af49c13cc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:40.100590 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d428f9-c938-4810-a849-dc00de8c8f15 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:40.112541 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d428f9-c938-4810-a849-dc00de8c8f15 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:40.118172 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d51d3-b77b-40ed-ac39-1f00af44e740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:40.118929 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d51d3-b77b-40ed-ac39-1f00af44e740 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:40.119213 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d51d3-b77b-40ed-ac39-1f00af44e740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:40.119485 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d51d3-b77b-40ed-ac39-1f00af44e740 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:40.120315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4d51d3-b77b-40ed-ac39-1f00af44e740 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:40.122645 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4d51d3-b77b-40ed-ac39-1f00af44e740 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:40.125414 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c7339733-87b6-4590-b45c-c98f782b24b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '857d05e10de645368a4cfe5af49c13cc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:40.127741 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c7339733-87b6-4590-b45c-c98f782b24b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '857d05e10de645368a4cfe5af49c13cc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:40.128968 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c7339733-87b6-4590-b45c-c98f782b24b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '857d05e10de645368a4cfe5af49c13cc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:40.142175 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c7339733-87b6-4590-b45c-c98f782b24b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '857d05e10de645368a4cfe5af49c13cc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:40.171504 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1040/2119] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:33:40 2022] POST /identity/v3/projects => generated 385 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:33:40.174715 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1080/2120] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:33:39 2022] DELETE /identity/v3/users/857d05e10de645368a4cfe5af49c13cc => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:33:40.179884 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d51d3-b77b-40ed-ac39-1f00af44e740 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:40.181975 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-aa4d51d3-b77b-40ed-ac39-1f00af44e740 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:40.217930 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7339733-87b6-4590-b45c-c98f782b24b7 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:40.223464 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c7339733-87b6-4590-b45c-c98f782b24b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:40.309735 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d51d3-b77b-40ed-ac39-1f00af44e740 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:40.387353 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7339733-87b6-4590-b45c-c98f782b24b7 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:40.395720 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be52c7-dd9d-4fb4-9357-a130ea7173e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:40.397323 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be52c7-dd9d-4fb4-9357-a130ea7173e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:40.397627 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be52c7-dd9d-4fb4-9357-a130ea7173e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:40.398080 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be52c7-dd9d-4fb4-9357-a130ea7173e5 admin admin] PATH_INFO: `/v3/projects/4b01a800e25c4d0c8f6391b96317064b` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:40.400389 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35be52c7-dd9d-4fb4-9357-a130ea7173e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b01a800e25c4d0c8f6391b96317064b)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:40.404523 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35be52c7-dd9d-4fb4-9357-a130ea7173e5 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:40.431079 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb3a0b-dd28-44f9-868f-198359bf207c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:40.431666 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb3a0b-dd28-44f9-868f-198359bf207c admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:40.432108 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb3a0b-dd28-44f9-868f-198359bf207c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:40.432392 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb3a0b-dd28-44f9-868f-198359bf207c admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:40.434569 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeb3a0b-dd28-44f9-868f-198359bf207c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:40.439091 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeb3a0b-dd28-44f9-868f-198359bf207c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:40.471748 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:40.490272 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1041/2121] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:33:40 2022] POST /identity/v3/users => generated 293 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:33:40.492093 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb3a0b-dd28-44f9-868f-198359bf207c admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:40.504362 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:40.518572 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb3a0b-dd28-44f9-868f-198359bf207c admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:40.523604 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-35be52c7-dd9d-4fb4-9357-a130ea7173e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b01a800e25c4d0c8f6391b96317064b', 'request_id': 'req-35be52c7-dd9d-4fb4-9357-a130ea7173e5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:40.533988 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-35be52c7-dd9d-4fb4-9357-a130ea7173e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b01a800e25c4d0c8f6391b96317064b', 'request_id': 'req-35be52c7-dd9d-4fb4-9357-a130ea7173e5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:40.545551 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-35be52c7-dd9d-4fb4-9357-a130ea7173e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b01a800e25c4d0c8f6391b96317064b', 'request_id': 'req-35be52c7-dd9d-4fb4-9357-a130ea7173e5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:40.571634 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-35be52c7-dd9d-4fb4-9357-a130ea7173e5 admin admin] The token cache is being invalidate because project 4b01a800e25c4d0c8f6391b96317064b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:33:40.572344 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-35be52c7-dd9d-4fb4-9357-a130ea7173e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:40.573622 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-35be52c7-dd9d-4fb4-9357-a130ea7173e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:40.576594 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1081/2122] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:33:40 2022] DELETE /identity/v3/projects/4b01a800e25c4d0c8f6391b96317064b => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:40.582779 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be52c7-dd9d-4fb4-9357-a130ea7173e5 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:40.585124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-35be52c7-dd9d-4fb4-9357-a130ea7173e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:40.631213 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5153a4-8058-4662-9260-746c6ba6571c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:40.632123 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5153a4-8058-4662-9260-746c6ba6571c admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:40.632844 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5153a4-8058-4662-9260-746c6ba6571c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:40.633396 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5153a4-8058-4662-9260-746c6ba6571c admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:40.635034 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5153a4-8058-4662-9260-746c6ba6571c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:40.638362 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5153a4-8058-4662-9260-746c6ba6571c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:40.662188 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1042/2123] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:33:40.668576 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5153a4-8058-4662-9260-746c6ba6571c admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:40.677224 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7c5153a4-8058-4662-9260-746c6ba6571c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:40.710336 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5153a4-8058-4662-9260-746c6ba6571c admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:40.713499 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be52c7-dd9d-4fb4-9357-a130ea7173e5 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:40.721946 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4554d21-3500-4fcc-a8d8-5da25c96de12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:40.723311 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4554d21-3500-4fcc-a8d8-5da25c96de12 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:40.723580 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1379532f-0933-42e1-a4f2-80d67e1479d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:40.723784 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4554d21-3500-4fcc-a8d8-5da25c96de12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:40.724226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4554d21-3500-4fcc-a8d8-5da25c96de12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:40.725131 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1379532f-0933-42e1-a4f2-80d67e1479d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:40.725569 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4554d21-3500-4fcc-a8d8-5da25c96de12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:40.725761 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1379532f-0933-42e1-a4f2-80d67e1479d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:40.726091 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1379532f-0933-42e1-a4f2-80d67e1479d3 admin admin] PATH_INFO: `/v3/users/b2f8d40a3f9347b999b485753af6ded6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:40.728285 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1379532f-0933-42e1-a4f2-80d67e1479d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2f8d40a3f9347b999b485753af6ded6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:40.728676 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4554d21-3500-4fcc-a8d8-5da25c96de12 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:40.731930 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1379532f-0933-42e1-a4f2-80d67e1479d3 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:40.754090 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1043/2124] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:33:40.754090 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4554d21-3500-4fcc-a8d8-5da25c96de12 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:40.754090 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b4554d21-3500-4fcc-a8d8-5da25c96de12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:40.769727 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4554d21-3500-4fcc-a8d8-5da25c96de12 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:40.778126 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff86a9-8186-48d5-a512-71200b0b2804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:40.779319 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff86a9-8186-48d5-a512-71200b0b2804 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:40.779660 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff86a9-8186-48d5-a512-71200b0b2804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:40.780156 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff86a9-8186-48d5-a512-71200b0b2804 admin admin] PATH_INFO: `/v3/projects/73c9075e9c4844d798e959056611266b` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:40.782068 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff86a9-8186-48d5-a512-71200b0b2804 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73c9075e9c4844d798e959056611266b)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:40.785072 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff86a9-8186-48d5-a512-71200b0b2804 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:40.799843 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1379532f-0933-42e1-a4f2-80d67e1479d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2f8d40a3f9347b999b485753af6ded6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:40.823380 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1379532f-0933-42e1-a4f2-80d67e1479d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2f8d40a3f9347b999b485753af6ded6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:40.824602 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1379532f-0933-42e1-a4f2-80d67e1479d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2f8d40a3f9347b999b485753af6ded6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:40.825852 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1379532f-0933-42e1-a4f2-80d67e1479d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2f8d40a3f9347b999b485753af6ded6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:40.838953 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1379532f-0933-42e1-a4f2-80d67e1479d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2f8d40a3f9347b999b485753af6ded6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:40.869118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8fff86a9-8186-48d5-a512-71200b0b2804 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73c9075e9c4844d798e959056611266b', 'request_id': 'req-8fff86a9-8186-48d5-a512-71200b0b2804'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:40.880801 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1082/2125] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:33:40 2022] DELETE /identity/v3/users/b2f8d40a3f9347b999b485753af6ded6 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:33:40.880801 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1379532f-0933-42e1-a4f2-80d67e1479d3 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:40.880801 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1379532f-0933-42e1-a4f2-80d67e1479d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:40.883422 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8fff86a9-8186-48d5-a512-71200b0b2804 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73c9075e9c4844d798e959056611266b', 'request_id': 'req-8fff86a9-8186-48d5-a512-71200b0b2804'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:40.900463 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8fff86a9-8186-48d5-a512-71200b0b2804 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73c9075e9c4844d798e959056611266b', 'request_id': 'req-8fff86a9-8186-48d5-a512-71200b0b2804'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:40.925220 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8fff86a9-8186-48d5-a512-71200b0b2804 admin admin] The token cache is being invalidate because project 73c9075e9c4844d798e959056611266b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:33:40.925797 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8fff86a9-8186-48d5-a512-71200b0b2804 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:40.926632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8fff86a9-8186-48d5-a512-71200b0b2804 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:40.928471 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1044/2126] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:33:40 2022] DELETE /identity/v3/projects/73c9075e9c4844d798e959056611266b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:40.934479 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bfc503-6b50-4ef9-9f0f-85111cddb6d3 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:40.934787 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bfc503-6b50-4ef9-9f0f-85111cddb6d3 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:40.935055 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bfc503-6b50-4ef9-9f0f-85111cddb6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:40.958951 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:40.989302 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1379532f-0933-42e1-a4f2-80d67e1479d3 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:40.998335 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ad9e25-463e-4e74-8ec2-5ee857c474d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:40.999777 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad9e25-463e-4e74-8ec2-5ee857c474d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:40.999777 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad9e25-463e-4e74-8ec2-5ee857c474d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:41.000016 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad9e25-463e-4e74-8ec2-5ee857c474d3 admin admin] PATH_INFO: `/v3/projects/6a6a20b925f64e7b97e4076342753d35/users/b16a88ecb9d24df29fc61a3f3866e318/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:41.002014 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-d4bfc503-6b50-4ef9-9f0f-85111cddb6d3 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:41.006820 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d4bfc503-6b50-4ef9-9f0f-85111cddb6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:41.022997 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1045/2127] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:33:41.039197 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ad9e25-463e-4e74-8ec2-5ee857c474d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a6a20b925f64e7b97e4076342753d35, user_id=b16a88ecb9d24df29fc61a3f3866e318, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:41.041813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a7e10-c897-4523-b2ca-b4fee2c1a927 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:41.042734 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ad9e25-463e-4e74-8ec2-5ee857c474d3 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:41.045377 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a7e10-c897-4523-b2ca-b4fee2c1a927 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:41.046093 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a7e10-c897-4523-b2ca-b4fee2c1a927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:41.086077 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1083/2128] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:33:40 2022] PUT /identity/v3/projects/6a6a20b925f64e7b97e4076342753d35/users/b16a88ecb9d24df29fc61a3f3866e318/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:41.107725 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de9b986-741d-4c60-b34a-ae1848df9d4b None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:41.108511 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de9b986-741d-4c60-b34a-ae1848df9d4b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:41.108999 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de9b986-741d-4c60-b34a-ae1848df9d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:41.120092 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-bf7a7e10-c897-4523-b2ca-b4fee2c1a927 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:41.127567 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-bf7a7e10-c897-4523-b2ca-b4fee2c1a927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:41.165943 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-3de9b986-741d-4c60-b34a-ae1848df9d4b None None] MFA Rules not processed for user `b16a88ecb9d24df29fc61a3f3866e318`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:41.168387 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3de9b986-741d-4c60-b34a-ae1848df9d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:41.189773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:41.336984 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1084/2129] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:33:41.341243 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1046/2130] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:41.348330 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de9b986-741d-4c60-b34a-ae1848df9d4b None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:41.353395 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a7e10-c897-4523-b2ca-b4fee2c1a927 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:41.355960 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-bf7a7e10-c897-4523-b2ca-b4fee2c1a927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:41.361255 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de9b986-741d-4c60-b34a-ae1848df9d4b None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:41.367278 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e18cf-2f3c-49c4-9cef-0b4e3e459672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:41.368071 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e18cf-2f3c-49c4-9cef-0b4e3e459672 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:41.368614 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e18cf-2f3c-49c4-9cef-0b4e3e459672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:41.368988 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e18cf-2f3c-49c4-9cef-0b4e3e459672 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:41.369890 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e18cf-2f3c-49c4-9cef-0b4e3e459672 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:33:41.370281 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e18cf-2f3c-49c4-9cef-0b4e3e459672 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:41.372336 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e18cf-2f3c-49c4-9cef-0b4e3e459672 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:41.392145 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1085/2131] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:33:41.399404 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e18cf-2f3c-49c4-9cef-0b4e3e459672 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:41.424606 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e18cf-2f3c-49c4-9cef-0b4e3e459672 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:41.437029 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d964d9a-8ec4-48c8-9a3b-f4de8b19ccfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:41.439213 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d964d9a-8ec4-48c8-9a3b-f4de8b19ccfd admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:41.441400 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d964d9a-8ec4-48c8-9a3b-f4de8b19ccfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:41.441738 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d964d9a-8ec4-48c8-9a3b-f4de8b19ccfd admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:41.445880 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d964d9a-8ec4-48c8-9a3b-f4de8b19ccfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:41.447973 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d964d9a-8ec4-48c8-9a3b-f4de8b19ccfd admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:41.493533 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1086/2132] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:33:41 2022] POST /identity/v3/projects => generated 391 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:33:41.494588 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a7e10-c897-4523-b2ca-b4fee2c1a927 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:41.500797 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d964d9a-8ec4-48c8-9a3b-f4de8b19ccfd admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:41.563297 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dda530-4243-474a-91bd-ad632b43fc26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:41.564239 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dda530-4243-474a-91bd-ad632b43fc26 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:41.564385 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dda530-4243-474a-91bd-ad632b43fc26 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:41.564682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dda530-4243-474a-91bd-ad632b43fc26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:41.646037 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d964d9a-8ec4-48c8-9a3b-f4de8b19ccfd admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:41.648914 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dda530-4243-474a-91bd-ad632b43fc26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:41.651805 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dda530-4243-474a-91bd-ad632b43fc26 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:41.654220 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985daa83-5701-4b2c-a0a9-3c4db8887063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:41.655499 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985daa83-5701-4b2c-a0a9-3c4db8887063 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:41.656090 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985daa83-5701-4b2c-a0a9-3c4db8887063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:41.656649 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985daa83-5701-4b2c-a0a9-3c4db8887063 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:41.658091 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985daa83-5701-4b2c-a0a9-3c4db8887063 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:41.662003 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985daa83-5701-4b2c-a0a9-3c4db8887063 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:41.669610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1047/2133] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:33:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:33:41.671001 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dda530-4243-474a-91bd-ad632b43fc26 service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:41.673230 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d1dda530-4243-474a-91bd-ad632b43fc26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:41.676106 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:41.702399 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dda530-4243-474a-91bd-ad632b43fc26 service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:41.709824 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1087/2134] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:33:41 2022] POST /identity/v3/users => generated 296 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:33:41.711717 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985daa83-5701-4b2c-a0a9-3c4db8887063 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:41.712054 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd1202-3df8-4817-a5b0-7133f19c2a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:41.713787 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd1202-3df8-4817-a5b0-7133f19c2a70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:41.714368 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd1202-3df8-4817-a5b0-7133f19c2a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:41.714916 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd1202-3df8-4817-a5b0-7133f19c2a70 admin admin] PATH_INFO: `/v3/projects/c87d5ce6ca6a4299aeba8d22c11eb44a` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:41.717184 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd1202-3df8-4817-a5b0-7133f19c2a70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c87d5ce6ca6a4299aeba8d22c11eb44a)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:41.721455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd1202-3df8-4817-a5b0-7133f19c2a70 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:41.732380 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985daa83-5701-4b2c-a0a9-3c4db8887063 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:41.741764 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395b704-6971-48b5-a7eb-e3ff4f25bd74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:41.742966 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395b704-6971-48b5-a7eb-e3ff4f25bd74 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:41.743452 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395b704-6971-48b5-a7eb-e3ff4f25bd74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:41.744029 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395b704-6971-48b5-a7eb-e3ff4f25bd74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:41.744450 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:41.745468 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3395b704-6971-48b5-a7eb-e3ff4f25bd74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:41.748497 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3395b704-6971-48b5-a7eb-e3ff4f25bd74 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:41.763244 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1088/2135] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 21:33:41.769049 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395b704-6971-48b5-a7eb-e3ff4f25bd74 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:41.785214 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395b704-6971-48b5-a7eb-e3ff4f25bd74 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:41.807342 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-aecd1202-3df8-4817-a5b0-7133f19c2a70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c87d5ce6ca6a4299aeba8d22c11eb44a', 'request_id': 'req-aecd1202-3df8-4817-a5b0-7133f19c2a70'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:41.821762 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-aecd1202-3df8-4817-a5b0-7133f19c2a70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c87d5ce6ca6a4299aeba8d22c11eb44a', 'request_id': 'req-aecd1202-3df8-4817-a5b0-7133f19c2a70'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:41.832764 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-aecd1202-3df8-4817-a5b0-7133f19c2a70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c87d5ce6ca6a4299aeba8d22c11eb44a', 'request_id': 'req-aecd1202-3df8-4817-a5b0-7133f19c2a70'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:41.859106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-aecd1202-3df8-4817-a5b0-7133f19c2a70 admin admin] The token cache is being invalidate because project c87d5ce6ca6a4299aeba8d22c11eb44a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:33:41.859806 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-aecd1202-3df8-4817-a5b0-7133f19c2a70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:41.861348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-aecd1202-3df8-4817-a5b0-7133f19c2a70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:41.870129 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1048/2136] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:33:41 2022] DELETE /identity/v3/projects/c87d5ce6ca6a4299aeba8d22c11eb44a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:41.874630 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bfca08-63c4-477c-a5b5-928d1e8fbc24 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:41.875337 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bfca08-63c4-477c-a5b5-928d1e8fbc24 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:41.875834 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bfca08-63c4-477c-a5b5-928d1e8fbc24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:41.907913 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7887bbbb-f7d6-4a97-9859-edc92cbb51dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:41.909195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887bbbb-f7d6-4a97-9859-edc92cbb51dd admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:41.909448 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-18bfca08-63c4-477c-a5b5-928d1e8fbc24 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:41.909711 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887bbbb-f7d6-4a97-9859-edc92cbb51dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:41.910184 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887bbbb-f7d6-4a97-9859-edc92cbb51dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:41.911552 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7887bbbb-f7d6-4a97-9859-edc92cbb51dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:41.911781 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-18bfca08-63c4-477c-a5b5-928d1e8fbc24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:41.914441 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7887bbbb-f7d6-4a97-9859-edc92cbb51dd admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:41.925607 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1049/2137] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:41.935556 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8ff0c-9007-455b-9906-2b1746ea80f4 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:41.936219 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8ff0c-9007-455b-9906-2b1746ea80f4 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:41.937472 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8ff0c-9007-455b-9906-2b1746ea80f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:41.938161 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1089/2138] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:33:41.949886 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7887bbbb-f7d6-4a97-9859-edc92cbb51dd admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:41.951931 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7887bbbb-f7d6-4a97-9859-edc92cbb51dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:41.966822 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7887bbbb-f7d6-4a97-9859-edc92cbb51dd admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:41.973371 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec257b-11fe-4119-9915-21dc817df1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:41.974594 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec257b-11fe-4119-9915-21dc817df1cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:41.975422 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec257b-11fe-4119-9915-21dc817df1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:41.975422 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec257b-11fe-4119-9915-21dc817df1cd admin admin] PATH_INFO: `/v3/projects/a69c8ac8a7d34c0fa6b1632a0c15ae7d/users/997d7aa08d444370a63e06b286d9fa66/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:41.983866 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b3b8ff0c-9007-455b-9906-2b1746ea80f4 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:41.986408 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b3b8ff0c-9007-455b-9906-2b1746ea80f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:41.996327 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec257b-11fe-4119-9915-21dc817df1cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a69c8ac8a7d34c0fa6b1632a0c15ae7d, user_id=997d7aa08d444370a63e06b286d9fa66, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:41.999185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec257b-11fe-4119-9915-21dc817df1cd admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:42.003306 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1050/2139] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:33:42.011250 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b8ff0c-9007-455b-9906-2b1746ea80f4 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:42.023701 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1090/2140] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:33:41 2022] PUT /identity/v3/projects/a69c8ac8a7d34c0fa6b1632a0c15ae7d/users/997d7aa08d444370a63e06b286d9fa66/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:42.028563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b8ff0c-9007-455b-9906-2b1746ea80f4 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:42.042187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f828d-0ebc-49d7-aac4-8a00941c55fb None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:42.042853 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f828d-0ebc-49d7-aac4-8a00941c55fb None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:42.043523 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f828d-0ebc-49d7-aac4-8a00941c55fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:42.099580 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9b3f828d-0ebc-49d7-aac4-8a00941c55fb None None] MFA Rules not processed for user `997d7aa08d444370a63e06b286d9fa66`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:42.101595 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9b3f828d-0ebc-49d7-aac4-8a00941c55fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:42.169218 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e7757d-4869-447e-a9b5-1077d11ef1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:42.170137 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7757d-4869-447e-a9b5-1077d11ef1f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:42.171136 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7757d-4869-447e-a9b5-1077d11ef1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:42.171945 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7757d-4869-447e-a9b5-1077d11ef1f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:42.174075 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e7757d-4869-447e-a9b5-1077d11ef1f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:33:42.174974 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e7757d-4869-447e-a9b5-1077d11ef1f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:42.179956 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e7757d-4869-447e-a9b5-1077d11ef1f8 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:42.208581 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1091/2141] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:42.214770 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1051/2142] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:33:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:33:42.219775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3f828d-0ebc-49d7-aac4-8a00941c55fb None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:42.220594 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e7757d-4869-447e-a9b5-1077d11ef1f8 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:42.243008 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e7757d-4869-447e-a9b5-1077d11ef1f8 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:42.254341 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3f828d-0ebc-49d7-aac4-8a00941c55fb None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:42.261047 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69411dd-b1cd-4a7d-87c7-eebe193f704c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:42.261714 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69411dd-b1cd-4a7d-87c7-eebe193f704c admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:42.262042 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69411dd-b1cd-4a7d-87c7-eebe193f704c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:42.262769 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69411dd-b1cd-4a7d-87c7-eebe193f704c admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:42.263402 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69411dd-b1cd-4a7d-87c7-eebe193f704c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:42.263750 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7468257e-3fed-4131-bd3f-832ce89b64a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:42.264479 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468257e-3fed-4131-bd3f-832ce89b64a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:42.264766 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468257e-3fed-4131-bd3f-832ce89b64a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:42.265047 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468257e-3fed-4131-bd3f-832ce89b64a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:42.265396 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69411dd-b1cd-4a7d-87c7-eebe193f704c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:42.266035 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7468257e-3fed-4131-bd3f-832ce89b64a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:42.271469 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7468257e-3fed-4131-bd3f-832ce89b64a5 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:42.306322 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1052/2143] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:33:42 2022] POST /identity/v3/projects => generated 387 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:33:42.311155 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69411dd-b1cd-4a7d-87c7-eebe193f704c admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:42.328063 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1092/2144] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:33:42 2022] POST /identity/v3/projects => generated 393 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:33:42.330767 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7468257e-3fed-4131-bd3f-832ce89b64a5 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:42.376440 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:42.460734 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7468257e-3fed-4131-bd3f-832ce89b64a5 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:42.467615 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc8a41-6fe7-4caf-8715-6f5587fe9ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:42.468960 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc8a41-6fe7-4caf-8715-6f5587fe9ef7 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:42.469737 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc8a41-6fe7-4caf-8715-6f5587fe9ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:42.470415 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc8a41-6fe7-4caf-8715-6f5587fe9ef7 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:42.471521 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:42.472066 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dc8a41-6fe7-4caf-8715-6f5587fe9ef7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:42.474139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dc8a41-6fe7-4caf-8715-6f5587fe9ef7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:42.481000 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69411dd-b1cd-4a7d-87c7-eebe193f704c admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:42.491824 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245a51b3-a0fe-4a78-a878-53995e0a9101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:42.493690 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a51b3-a0fe-4a78-a878-53995e0a9101 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:42.494698 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a51b3-a0fe-4a78-a878-53995e0a9101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:42.496167 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a51b3-a0fe-4a78-a878-53995e0a9101 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:42.497645 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245a51b3-a0fe-4a78-a878-53995e0a9101 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:42.502795 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245a51b3-a0fe-4a78-a878-53995e0a9101 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:42.506523 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1093/2145] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:33:42 2022] POST /identity/v3/users => generated 303 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:33:42.513182 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc8a41-6fe7-4caf-8715-6f5587fe9ef7 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:42.529861 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc8a41-6fe7-4caf-8715-6f5587fe9ef7 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:42.545218 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d49880-50d2-4f70-a9b3-3cf30d7505f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:42.546092 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d49880-50d2-4f70-a9b3-3cf30d7505f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:42.546855 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d49880-50d2-4f70-a9b3-3cf30d7505f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:42.547380 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d49880-50d2-4f70-a9b3-3cf30d7505f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:42.549314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d49880-50d2-4f70-a9b3-3cf30d7505f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:42.552434 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d49880-50d2-4f70-a9b3-3cf30d7505f9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:42.555627 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1053/2146] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:33:42 2022] POST /identity/v3/users => generated 294 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:33:42.564624 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245a51b3-a0fe-4a78-a878-53995e0a9101 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:42.577025 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1094/2147] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:33:42.579166 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d49880-50d2-4f70-a9b3-3cf30d7505f9 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:42.585420 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245a51b3-a0fe-4a78-a878-53995e0a9101 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:42.592860 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2adbac-7017-4e11-90c6-c098c724f04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:42.594167 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2adbac-7017-4e11-90c6-c098c724f04f admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:42.595008 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2adbac-7017-4e11-90c6-c098c724f04f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:42.595673 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2adbac-7017-4e11-90c6-c098c724f04f admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:42.596561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2adbac-7017-4e11-90c6-c098c724f04f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:42.598379 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2adbac-7017-4e11-90c6-c098c724f04f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:42.604880 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d49880-50d2-4f70-a9b3-3cf30d7505f9 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:42.614002 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b341b6c-87d3-4ab3-a4fc-8537448245c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:42.615850 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b341b6c-87d3-4ab3-a4fc-8537448245c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:42.616389 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b341b6c-87d3-4ab3-a4fc-8537448245c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:42.616955 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b341b6c-87d3-4ab3-a4fc-8537448245c6 admin admin] PATH_INFO: `/v3/projects/d6fbdeaef61748ebbe25e365ec81f561/users/be4d35be6a2b48b5a750ef53b6490933/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:42.617775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1054/2148] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:33:42.622270 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2adbac-7017-4e11-90c6-c098c724f04f admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:42.634391 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2adbac-7017-4e11-90c6-c098c724f04f admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:42.640105 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4180d-5a8e-45ad-8c73-741d074d400c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:42.640799 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4180d-5a8e-45ad-8c73-741d074d400c admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:42.641083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4180d-5a8e-45ad-8c73-741d074d400c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:42.641523 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4180d-5a8e-45ad-8c73-741d074d400c admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:42.642776 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f4180d-5a8e-45ad-8c73-741d074d400c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:42.644743 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f4180d-5a8e-45ad-8c73-741d074d400c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:42.651105 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b341b6c-87d3-4ab3-a4fc-8537448245c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6fbdeaef61748ebbe25e365ec81f561, user_id=be4d35be6a2b48b5a750ef53b6490933, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:42.653997 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b341b6c-87d3-4ab3-a4fc-8537448245c6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:42.660698 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1055/2149] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 21:33:42.667025 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4180d-5a8e-45ad-8c73-741d074d400c admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:42.683309 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1095/2150] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:33:42 2022] PUT /identity/v3/projects/d6fbdeaef61748ebbe25e365ec81f561/users/be4d35be6a2b48b5a750ef53b6490933/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:42.687623 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4180d-5a8e-45ad-8c73-741d074d400c admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:42.688911 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b341b6c-87d3-4ab3-a4fc-8537448245c6 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:42.823697 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8232704-0ce3-4e97-9ef1-52c4fb35bd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:42.826392 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8232704-0ce3-4e97-9ef1-52c4fb35bd6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:42.827013 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8232704-0ce3-4e97-9ef1-52c4fb35bd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:42.827625 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8232704-0ce3-4e97-9ef1-52c4fb35bd6a admin admin] PATH_INFO: `/v3/projects/c82e8c75d0e8492fb61f3136b4bfcfd9/users/d1923a105dad4578b2a08adbc1bc17d9/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:42.846892 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b341b6c-87d3-4ab3-a4fc-8537448245c6 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:42.852637 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a50bc-6a89-4a84-a31e-c85761a3262d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:42.853383 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a50bc-6a89-4a84-a31e-c85761a3262d admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:42.853690 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a50bc-6a89-4a84-a31e-c85761a3262d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:42.853995 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a50bc-6a89-4a84-a31e-c85761a3262d admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:42.854900 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a50bc-6a89-4a84-a31e-c85761a3262d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:42.856907 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a50bc-6a89-4a84-a31e-c85761a3262d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:42.861507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8232704-0ce3-4e97-9ef1-52c4fb35bd6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c82e8c75d0e8492fb61f3136b4bfcfd9, user_id=d1923a105dad4578b2a08adbc1bc17d9, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:42.867261 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8232704-0ce3-4e97-9ef1-52c4fb35bd6a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:42.870657 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1096/2151] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 21:33:42.877996 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a50bc-6a89-4a84-a31e-c85761a3262d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:42.903368 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1056/2152] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:33:42 2022] PUT /identity/v3/projects/c82e8c75d0e8492fb61f3136b4bfcfd9/users/d1923a105dad4578b2a08adbc1bc17d9/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:33:42.903599 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a50bc-6a89-4a84-a31e-c85761a3262d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:42.906651 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ebdba-2051-4e85-8758-3fca1032da1d None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:42.906961 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ebdba-2051-4e85-8758-3fca1032da1d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:42.907430 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ebdba-2051-4e85-8758-3fca1032da1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:42.944608 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-a83ebdba-2051-4e85-8758-3fca1032da1d None None] MFA Rules not processed for user `d1923a105dad4578b2a08adbc1bc17d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:42.947433 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a83ebdba-2051-4e85-8758-3fca1032da1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:42.991407 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a045aff0-6ce6-434c-80d7-4e0a5a7d50a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:42.992647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045aff0-6ce6-434c-80d7-4e0a5a7d50a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:42.993029 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045aff0-6ce6-434c-80d7-4e0a5a7d50a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:42.993437 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045aff0-6ce6-434c-80d7-4e0a5a7d50a9 admin admin] PATH_INFO: `/v3/projects/d6fbdeaef61748ebbe25e365ec81f561/users/be4d35be6a2b48b5a750ef53b6490933/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:42.996447 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a045aff0-6ce6-434c-80d7-4e0a5a7d50a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6fbdeaef61748ebbe25e365ec81f561, user_id=be4d35be6a2b48b5a750ef53b6490933, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:42.999815 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a045aff0-6ce6-434c-80d7-4e0a5a7d50a9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:43.018694 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1097/2153] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:33:42 2022] PUT /identity/v3/projects/d6fbdeaef61748ebbe25e365ec81f561/users/be4d35be6a2b48b5a750ef53b6490933/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:33:43.033771 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a420bc7-9672-4fa8-887a-bb5b41dbb2b6 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:43.034329 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a420bc7-9672-4fa8-887a-bb5b41dbb2b6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:43.034899 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a420bc7-9672-4fa8-887a-bb5b41dbb2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:43.047276 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:43.058758 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1057/2154] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:33:43.061367 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ebdba-2051-4e85-8758-3fca1032da1d None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:43.083267 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ebdba-2051-4e85-8758-3fca1032da1d None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:43.099151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0a420bc7-9672-4fa8-887a-bb5b41dbb2b6 None None] MFA Rules not processed for user `be4d35be6a2b48b5a750ef53b6490933`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:43.101008 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0a420bc7-9672-4fa8-887a-bb5b41dbb2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:43.137906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:43.171638 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa772f9-71d6-4d24-9e43-120e0b16c30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:43.173192 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa772f9-71d6-4d24-9e43-120e0b16c30e admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:43.173694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa772f9-71d6-4d24-9e43-120e0b16c30e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:43.174121 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa772f9-71d6-4d24-9e43-120e0b16c30e admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:43.175397 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa772f9-71d6-4d24-9e43-120e0b16c30e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:43.178967 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa772f9-71d6-4d24-9e43-120e0b16c30e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:43.206385 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1098/2155] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:43 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:43.222503 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1058/2156] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:33:43 2022] POST /identity/v3/projects => generated 385 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:33:43.223388 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a420bc7-9672-4fa8-887a-bb5b41dbb2b6 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:43.225463 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa772f9-71d6-4d24-9e43-120e0b16c30e admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:43.225705 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0a420bc7-9672-4fa8-887a-bb5b41dbb2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:43.313556 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a420bc7-9672-4fa8-887a-bb5b41dbb2b6 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:43.315635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa772f9-71d6-4d24-9e43-120e0b16c30e admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:43.321312 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce2c2b-ecc8-44e1-b2fe-d7a4aba18098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:43.322006 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce2c2b-ecc8-44e1-b2fe-d7a4aba18098 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:43.322490 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce2c2b-ecc8-44e1-b2fe-d7a4aba18098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:43.322772 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce2c2b-ecc8-44e1-b2fe-d7a4aba18098 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:43.323511 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce2c2b-ecc8-44e1-b2fe-d7a4aba18098 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:43.325529 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce2c2b-ecc8-44e1-b2fe-d7a4aba18098 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:43.359194 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1059/2157] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:33:43 2022] POST /identity/v3/users => generated 299 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:33:43.361415 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce2c2b-ecc8-44e1-b2fe-d7a4aba18098 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:43.373312 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce2c2b-ecc8-44e1-b2fe-d7a4aba18098 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:43.378635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d5edd-1aac-48c4-a1ea-1327bc8801ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:43.379326 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d5edd-1aac-48c4-a1ea-1327bc8801ba admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:43.379792 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d5edd-1aac-48c4-a1ea-1327bc8801ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:43.379955 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d5edd-1aac-48c4-a1ea-1327bc8801ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:43.380615 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55d5edd-1aac-48c4-a1ea-1327bc8801ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:43.382337 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55d5edd-1aac-48c4-a1ea-1327bc8801ba admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:43.386499 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b42e3e-bfce-4de8-ac9d-5d6ac042f987 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:43.387296 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b42e3e-bfce-4de8-ac9d-5d6ac042f987 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:43.387697 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b42e3e-bfce-4de8-ac9d-5d6ac042f987 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:43.388061 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b42e3e-bfce-4de8-ac9d-5d6ac042f987 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:43.397387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1060/2158] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 21:33:43.403573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d5edd-1aac-48c4-a1ea-1327bc8801ba admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:43.417619 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d5edd-1aac-48c4-a1ea-1327bc8801ba admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:43.427323 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27f976d-07fc-4174-ac3f-df1c88145d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:43.429232 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f976d-07fc-4174-ac3f-df1c88145d5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:43.429696 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f976d-07fc-4174-ac3f-df1c88145d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:43.430167 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f976d-07fc-4174-ac3f-df1c88145d5b admin admin] PATH_INFO: `/v3/projects/7e7fd7fcbcfb484581f469cce9f9ed19/users/c374b038bbb64cf587c6882aedc69949/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:43.456442 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27f976d-07fc-4174-ac3f-df1c88145d5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e7fd7fcbcfb484581f469cce9f9ed19, user_id=c374b038bbb64cf587c6882aedc69949, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:43.459318 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27f976d-07fc-4174-ac3f-df1c88145d5b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:43.477763 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1061/2159] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:33:43 2022] PUT /identity/v3/projects/7e7fd7fcbcfb484581f469cce9f9ed19/users/c374b038bbb64cf587c6882aedc69949/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:33:43.480340 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:43.482570 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27f976d-07fc-4174-ac3f-df1c88145d5b admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:43.485622 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b42e3e-bfce-4de8-ac9d-5d6ac042f987 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:43.487578 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b42e3e-bfce-4de8-ac9d-5d6ac042f987 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:43.503538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1099/2160] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:33:43 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:33:43.622012 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27f976d-07fc-4174-ac3f-df1c88145d5b admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:43.634655 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b248c2d-8272-40c5-a1fe-8ea3f3750da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:43.636879 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b248c2d-8272-40c5-a1fe-8ea3f3750da2 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:43.637307 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b248c2d-8272-40c5-a1fe-8ea3f3750da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:43.637953 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b248c2d-8272-40c5-a1fe-8ea3f3750da2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:43.639907 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b248c2d-8272-40c5-a1fe-8ea3f3750da2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:43.644884 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b248c2d-8272-40c5-a1fe-8ea3f3750da2 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:43.655390 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b42e3e-bfce-4de8-ac9d-5d6ac042f987 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:43.673938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1062/2161] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:43 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 21:33:43.678146 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b42e3e-bfce-4de8-ac9d-5d6ac042f987 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:43.680869 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b248c2d-8272-40c5-a1fe-8ea3f3750da2 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:43.706006 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b248c2d-8272-40c5-a1fe-8ea3f3750da2 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:43.716382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502204cb-679e-4419-a2c2-a288d583e563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:43.718171 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502204cb-679e-4419-a2c2-a288d583e563 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:43.719200 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502204cb-679e-4419-a2c2-a288d583e563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:43.719782 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502204cb-679e-4419-a2c2-a288d583e563 admin admin] PATH_INFO: `/v3/projects/7e7fd7fcbcfb484581f469cce9f9ed19/users/c374b038bbb64cf587c6882aedc69949/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:43.725445 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502204cb-679e-4419-a2c2-a288d583e563 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e7fd7fcbcfb484581f469cce9f9ed19, user_id=c374b038bbb64cf587c6882aedc69949, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:43.731465 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502204cb-679e-4419-a2c2-a288d583e563 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:43.750457 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:43.763434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1063/2162] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:33:43 2022] PUT /identity/v3/projects/7e7fd7fcbcfb484581f469cce9f9ed19/users/c374b038bbb64cf587c6882aedc69949/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:33:43.788751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad25a21-86a6-4714-ad11-32f1b6358156 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:43.789248 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad25a21-86a6-4714-ad11-32f1b6358156 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:43.789735 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad25a21-86a6-4714-ad11-32f1b6358156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:43.829999 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd310a9a-c554-40e3-bcb2-e0bda058675a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:43.830801 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd310a9a-c554-40e3-bcb2-e0bda058675a service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:43.831310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd310a9a-c554-40e3-bcb2-e0bda058675a service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:43.831609 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd310a9a-c554-40e3-bcb2-e0bda058675a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:43.859007 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-3ad25a21-86a6-4714-ad11-32f1b6358156 None None] MFA Rules not processed for user `c374b038bbb64cf587c6882aedc69949`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:43.861222 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3ad25a21-86a6-4714-ad11-32f1b6358156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:43.916267 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd310a9a-c554-40e3-bcb2-e0bda058675a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:43.917804 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd310a9a-c554-40e3-bcb2-e0bda058675a service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:43.933396 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1100/2163] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:33:43 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:33:44.005298 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1064/2164] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:43 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:44.026617 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd310a9a-c554-40e3-bcb2-e0bda058675a service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:44.029005 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cd310a9a-c554-40e3-bcb2-e0bda058675a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:44.051795 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd310a9a-c554-40e3-bcb2-e0bda058675a service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:44.163373 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf4d4f-fac2-4049-aa02-70ea7dbbd937 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:44.164656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf4d4f-fac2-4049-aa02-70ea7dbbd937 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:44.165306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf4d4f-fac2-4049-aa02-70ea7dbbd937 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:44.165934 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf4d4f-fac2-4049-aa02-70ea7dbbd937 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:44.187315 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cf4d4f-fac2-4049-aa02-70ea7dbbd937 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:44.192075 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cf4d4f-fac2-4049-aa02-70ea7dbbd937 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:44.214831 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1101/2165] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:33:44 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:33:44.256927 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad25a21-86a6-4714-ad11-32f1b6358156 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:44.362536 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad25a21-86a6-4714-ad11-32f1b6358156 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:44.371223 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0a7a5-5bc2-4e78-bcf6-953f302208e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:44.372714 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0a7a5-5bc2-4e78-bcf6-953f302208e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:44.373216 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0a7a5-5bc2-4e78-bcf6-953f302208e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:44.373642 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0a7a5-5bc2-4e78-bcf6-953f302208e0 admin admin] PATH_INFO: `/v3/users/d1923a105dad4578b2a08adbc1bc17d9` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:44.376272 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a0a7a5-5bc2-4e78-bcf6-953f302208e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1923a105dad4578b2a08adbc1bc17d9)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:44.380064 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a0a7a5-5bc2-4e78-bcf6-953f302208e0 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:44.461568 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a4a0a7a5-5bc2-4e78-bcf6-953f302208e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1923a105dad4578b2a08adbc1bc17d9'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:44.462273 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a4a0a7a5-5bc2-4e78-bcf6-953f302208e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1923a105dad4578b2a08adbc1bc17d9'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:44.478545 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a4a0a7a5-5bc2-4e78-bcf6-953f302208e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1923a105dad4578b2a08adbc1bc17d9'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:44.488490 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a4a0a7a5-5bc2-4e78-bcf6-953f302208e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1923a105dad4578b2a08adbc1bc17d9'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:44.498610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a4a0a7a5-5bc2-4e78-bcf6-953f302208e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1923a105dad4578b2a08adbc1bc17d9'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:44.508009 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1065/2166] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:33:44 2022] DELETE /identity/v3/users/d1923a105dad4578b2a08adbc1bc17d9 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:44.513468 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ff96f-8b91-493a-b60b-c31668bad0b3 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:44.513896 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ff96f-8b91-493a-b60b-c31668bad0b3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:44.514402 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ff96f-8b91-493a-b60b-c31668bad0b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:44.536537 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-855ff96f-8b91-493a-b60b-c31668bad0b3 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:44.538129 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-855ff96f-8b91-493a-b60b-c31668bad0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:44.599621 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:44.621940 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1102/2167] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:44.633188 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0a7a5-5bc2-4e78-bcf6-953f302208e0 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:44.635669 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a4a0a7a5-5bc2-4e78-bcf6-953f302208e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:44.644392 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:44.772056 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0a7a5-5bc2-4e78-bcf6-953f302208e0 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:44.777693 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf393a3-bb14-4038-b02c-a04c2e1db8d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:44.778848 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf393a3-bb14-4038-b02c-a04c2e1db8d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:44.779311 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf393a3-bb14-4038-b02c-a04c2e1db8d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:44.779776 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf393a3-bb14-4038-b02c-a04c2e1db8d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:44.798167 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf393a3-bb14-4038-b02c-a04c2e1db8d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:44.800197 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf393a3-bb14-4038-b02c-a04c2e1db8d3 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:44.816185 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1066/2168] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:33:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:33:45.245383 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855ff96f-8b91-493a-b60b-c31668bad0b3 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:45.247180 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-855ff96f-8b91-493a-b60b-c31668bad0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:45.265627 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855ff96f-8b91-493a-b60b-c31668bad0b3 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:45.275809 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100f4a0-9335-4390-b17d-c9d4178b67d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:45.277836 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100f4a0-9335-4390-b17d-c9d4178b67d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:45.278619 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100f4a0-9335-4390-b17d-c9d4178b67d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:45.279312 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100f4a0-9335-4390-b17d-c9d4178b67d9 admin admin] PATH_INFO: `/v3/projects/c82e8c75d0e8492fb61f3136b4bfcfd9` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:45.284391 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b100f4a0-9335-4390-b17d-c9d4178b67d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c82e8c75d0e8492fb61f3136b4bfcfd9)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:45.288050 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b100f4a0-9335-4390-b17d-c9d4178b67d9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:45.371205 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b100f4a0-9335-4390-b17d-c9d4178b67d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c82e8c75d0e8492fb61f3136b4bfcfd9', 'request_id': 'req-b100f4a0-9335-4390-b17d-c9d4178b67d9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:45.380699 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b100f4a0-9335-4390-b17d-c9d4178b67d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c82e8c75d0e8492fb61f3136b4bfcfd9', 'request_id': 'req-b100f4a0-9335-4390-b17d-c9d4178b67d9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:45.390582 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b100f4a0-9335-4390-b17d-c9d4178b67d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c82e8c75d0e8492fb61f3136b4bfcfd9', 'request_id': 'req-b100f4a0-9335-4390-b17d-c9d4178b67d9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:45.432045 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b100f4a0-9335-4390-b17d-c9d4178b67d9 admin admin] The token cache is being invalidate because project c82e8c75d0e8492fb61f3136b4bfcfd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:33:45.432374 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b100f4a0-9335-4390-b17d-c9d4178b67d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:45.433267 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b100f4a0-9335-4390-b17d-c9d4178b67d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:45.438396 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1103/2169] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:33:45 2022] DELETE /identity/v3/projects/c82e8c75d0e8492fb61f3136b4bfcfd9 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:45.443733 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf393a3-bb14-4038-b02c-a04c2e1db8d3 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:45.461928 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf393a3-bb14-4038-b02c-a04c2e1db8d3 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:45.591858 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c4bd10-edc5-4992-9572-c12620b802a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:45.593346 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c4bd10-edc5-4992-9572-c12620b802a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:45.594434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c4bd10-edc5-4992-9572-c12620b802a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:45.595027 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c4bd10-edc5-4992-9572-c12620b802a4 admin admin] PATH_INFO: `/v3/users/c374b038bbb64cf587c6882aedc69949` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:45.597135 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c4bd10-edc5-4992-9572-c12620b802a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c374b038bbb64cf587c6882aedc69949)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:45.600621 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c4bd10-edc5-4992-9572-c12620b802a4 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:45.680146 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-25c4bd10-edc5-4992-9572-c12620b802a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c374b038bbb64cf587c6882aedc69949'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:45.681479 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-25c4bd10-edc5-4992-9572-c12620b802a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c374b038bbb64cf587c6882aedc69949'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:45.705523 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-25c4bd10-edc5-4992-9572-c12620b802a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c374b038bbb64cf587c6882aedc69949'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:45.721253 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-25c4bd10-edc5-4992-9572-c12620b802a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c374b038bbb64cf587c6882aedc69949'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:45.733672 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-25c4bd10-edc5-4992-9572-c12620b802a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c374b038bbb64cf587c6882aedc69949'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:45.746956 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1067/2170] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:33:45 2022] DELETE /identity/v3/users/c374b038bbb64cf587c6882aedc69949 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:46.342204 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100f4a0-9335-4390-b17d-c9d4178b67d9 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:46.346375 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b100f4a0-9335-4390-b17d-c9d4178b67d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:46.357448 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c4bd10-edc5-4992-9572-c12620b802a4 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:46.359859 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-25c4bd10-edc5-4992-9572-c12620b802a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:46.425494 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:46.542099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100f4a0-9335-4390-b17d-c9d4178b67d9 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:46.744215 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c4bd10-edc5-4992-9572-c12620b802a4 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:46.777995 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb89661-982a-4fa1-8564-1792d08bf88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:46.780119 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb89661-982a-4fa1-8564-1792d08bf88d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:46.782068 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb89661-982a-4fa1-8564-1792d08bf88d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:46.783775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb89661-982a-4fa1-8564-1792d08bf88d admin admin] PATH_INFO: `/v3/projects/7e7fd7fcbcfb484581f469cce9f9ed19` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:46.786379 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb89661-982a-4fa1-8564-1792d08bf88d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e7fd7fcbcfb484581f469cce9f9ed19)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:46.801038 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb89661-982a-4fa1-8564-1792d08bf88d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:46.831524 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1c9a1-9caf-4148-a301-601ea78eb196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:46.840515 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1c9a1-9caf-4148-a301-601ea78eb196 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:46.841121 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1c9a1-9caf-4148-a301-601ea78eb196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:46.841884 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1c9a1-9caf-4148-a301-601ea78eb196 admin admin] PATH_INFO: `/v3/users/b16a88ecb9d24df29fc61a3f3866e318` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:46.848351 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1c9a1-9caf-4148-a301-601ea78eb196 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b16a88ecb9d24df29fc61a3f3866e318)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:46.862045 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1c9a1-9caf-4148-a301-601ea78eb196 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:46.939718 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0eb89661-982a-4fa1-8564-1792d08bf88d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e7fd7fcbcfb484581f469cce9f9ed19', 'request_id': 'req-0eb89661-982a-4fa1-8564-1792d08bf88d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:46.948330 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0eb89661-982a-4fa1-8564-1792d08bf88d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e7fd7fcbcfb484581f469cce9f9ed19', 'request_id': 'req-0eb89661-982a-4fa1-8564-1792d08bf88d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:46.962221 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0eb89661-982a-4fa1-8564-1792d08bf88d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e7fd7fcbcfb484581f469cce9f9ed19', 'request_id': 'req-0eb89661-982a-4fa1-8564-1792d08bf88d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:46.984167 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0eb89661-982a-4fa1-8564-1792d08bf88d admin admin] The token cache is being invalidate because project 7e7fd7fcbcfb484581f469cce9f9ed19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:33:46.984794 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0eb89661-982a-4fa1-8564-1792d08bf88d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:46.985701 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0eb89661-982a-4fa1-8564-1792d08bf88d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:46.988935 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1068/2171] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:33:46 2022] DELETE /identity/v3/projects/7e7fd7fcbcfb484581f469cce9f9ed19 => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:46.998685 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4a3b1-f702-44cc-9e1e-db9ac721f1b5 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:46.999139 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4a3b1-f702-44cc-9e1e-db9ac721f1b5 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:46.999651 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4a3b1-f702-44cc-9e1e-db9ac721f1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:47.055246 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-98f1c9a1-9caf-4148-a301-601ea78eb196 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b16a88ecb9d24df29fc61a3f3866e318'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:47.087844 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-7dc4a3b1-f702-44cc-9e1e-db9ac721f1b5 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:47.089583 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7dc4a3b1-f702-44cc-9e1e-db9ac721f1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:47.098264 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:47.105151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-98f1c9a1-9caf-4148-a301-601ea78eb196 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b16a88ecb9d24df29fc61a3f3866e318'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:47.109888 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-98f1c9a1-9caf-4148-a301-601ea78eb196 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b16a88ecb9d24df29fc61a3f3866e318'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:47.113458 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-98f1c9a1-9caf-4148-a301-601ea78eb196 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b16a88ecb9d24df29fc61a3f3866e318'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:47.137635 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-98f1c9a1-9caf-4148-a301-601ea78eb196 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b16a88ecb9d24df29fc61a3f3866e318'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:47.195277 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1104/2172] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:33:46 2022] DELETE /identity/v3/users/b16a88ecb9d24df29fc61a3f3866e318 => generated 0 bytes in 851 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:47.198025 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae1cd2-873d-4be9-b174-c50d5e5fa46a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:47.198769 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae1cd2-873d-4be9-b174-c50d5e5fa46a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:47.199351 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae1cd2-873d-4be9-b174-c50d5e5fa46a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:47.226699 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-cdae1cd2-873d-4be9-b174-c50d5e5fa46a None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:47.228334 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cdae1cd2-873d-4be9-b174-c50d5e5fa46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:47.236306 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1069/2173] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:33:47.250372 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad605cfe-41dd-469f-928a-11b92e146c7c None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:47.251278 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad605cfe-41dd-469f-928a-11b92e146c7c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:47.251660 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad605cfe-41dd-469f-928a-11b92e146c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:47.280066 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-ad605cfe-41dd-469f-928a-11b92e146c7c None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:47.283054 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ad605cfe-41dd-469f-928a-11b92e146c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:47.291930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:47.355764 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1105/2174] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:47.368659 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdae1cd2-873d-4be9-b174-c50d5e5fa46a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:47.370864 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cdae1cd2-873d-4be9-b174-c50d5e5fa46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:47.396005 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1070/2175] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:33:47.400509 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad605cfe-41dd-469f-928a-11b92e146c7c None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:47.413330 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad605cfe-41dd-469f-928a-11b92e146c7c None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:47.419620 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6567da4-2a33-478c-8056-a2102c220ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:47.420388 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6567da4-2a33-478c-8056-a2102c220ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:47.420741 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6567da4-2a33-478c-8056-a2102c220ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:47.421074 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6567da4-2a33-478c-8056-a2102c220ce8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:47.422136 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6567da4-2a33-478c-8056-a2102c220ce8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:33:47.422733 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6567da4-2a33-478c-8056-a2102c220ce8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:47.425032 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6567da4-2a33-478c-8056-a2102c220ce8 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:47.446377 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1071/2176] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:33:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:33:47.446650 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdae1cd2-873d-4be9-b174-c50d5e5fa46a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:47.451729 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6567da4-2a33-478c-8056-a2102c220ce8 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:47.451982 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda07ce-f1fd-4ccd-888c-fa5539afb2e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:47.452641 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda07ce-f1fd-4ccd-888c-fa5539afb2e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:47.453196 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda07ce-f1fd-4ccd-888c-fa5539afb2e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:47.453615 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda07ce-f1fd-4ccd-888c-fa5539afb2e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:47.465380 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6567da4-2a33-478c-8056-a2102c220ce8 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:47.467224 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eda07ce-f1fd-4ccd-888c-fa5539afb2e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:47.469666 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eda07ce-f1fd-4ccd-888c-fa5539afb2e2 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:47.471614 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c81927-d594-464b-9fed-e5c30c53d8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:47.472506 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c81927-d594-464b-9fed-e5c30c53d8ab admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:47.472791 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c81927-d594-464b-9fed-e5c30c53d8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:47.473129 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c81927-d594-464b-9fed-e5c30c53d8ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:47.474619 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c81927-d594-464b-9fed-e5c30c53d8ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:47.476639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c81927-d594-464b-9fed-e5c30c53d8ab admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:47.483449 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1106/2177] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:33:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:33:47.523701 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1072/2178] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:33:47 2022] POST /identity/v3/projects => generated 371 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:33:47.527529 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda07ce-f1fd-4ccd-888c-fa5539afb2e2 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:47.544911 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda07ce-f1fd-4ccd-888c-fa5539afb2e2 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:47.652413 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa7bc53-2063-4fb2-8c54-ddc18fcd96ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:47.654224 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7bc53-2063-4fb2-8c54-ddc18fcd96ff admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:47.654224 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7bc53-2063-4fb2-8c54-ddc18fcd96ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:47.654544 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7bc53-2063-4fb2-8c54-ddc18fcd96ff admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:47.655494 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa7bc53-2063-4fb2-8c54-ddc18fcd96ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:47.657622 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa7bc53-2063-4fb2-8c54-ddc18fcd96ff admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:47.682951 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1107/2179] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:33:47 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:33:47.689711 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c81927-d594-464b-9fed-e5c30c53d8ab admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:47.703988 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c81927-d594-464b-9fed-e5c30c53d8ab admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:47.709455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48532a7b-ffe4-4158-957d-26379f6e3d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:47.710182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48532a7b-ffe4-4158-957d-26379f6e3d62 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:47.710526 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48532a7b-ffe4-4158-957d-26379f6e3d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:47.710801 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48532a7b-ffe4-4158-957d-26379f6e3d62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:47.711600 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48532a7b-ffe4-4158-957d-26379f6e3d62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:47.713361 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48532a7b-ffe4-4158-957d-26379f6e3d62 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:47.731302 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1073/2180] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:33:47.733759 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa7bc53-2063-4fb2-8c54-ddc18fcd96ff admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:47.745098 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa7bc53-2063-4fb2-8c54-ddc18fcd96ff admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:47.750462 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0b6fb7-9947-4df8-8de7-b37ed1c7cddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:47.751237 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b6fb7-9947-4df8-8de7-b37ed1c7cddd admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:47.751580 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b6fb7-9947-4df8-8de7-b37ed1c7cddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:47.751817 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b6fb7-9947-4df8-8de7-b37ed1c7cddd admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:47.752549 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0b6fb7-9947-4df8-8de7-b37ed1c7cddd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:47.754419 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0b6fb7-9947-4df8-8de7-b37ed1c7cddd admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:47.760860 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:47.767328 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1108/2181] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:33:47.782473 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48532a7b-ffe4-4158-957d-26379f6e3d62 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:47.799166 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48532a7b-ffe4-4158-957d-26379f6e3d62 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:47.808595 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d2554-ea55-4600-851b-f8e49bfacf0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:47.809642 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d2554-ea55-4600-851b-f8e49bfacf0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:47.809921 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d2554-ea55-4600-851b-f8e49bfacf0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:47.810198 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d2554-ea55-4600-851b-f8e49bfacf0c admin admin] PATH_INFO: `/v3/projects/2fb03711de75418e89907ca315beaadc/users/c5dc7ff3a5624d8ca2cf116f8f9f0b2e/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:47.831392 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60d2554-ea55-4600-851b-f8e49bfacf0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fb03711de75418e89907ca315beaadc, user_id=c5dc7ff3a5624d8ca2cf116f8f9f0b2e, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:47.834195 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60d2554-ea55-4600-851b-f8e49bfacf0c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:47.867101 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1074/2182] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:33:47 2022] PUT /identity/v3/projects/2fb03711de75418e89907ca315beaadc/users/c5dc7ff3a5624d8ca2cf116f8f9f0b2e/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:47.874203 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02923bdf-c010-4373-b57b-7612a4002f5a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:47.874623 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02923bdf-c010-4373-b57b-7612a4002f5a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:47.874883 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02923bdf-c010-4373-b57b-7612a4002f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:47.898072 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-02923bdf-c010-4373-b57b-7612a4002f5a None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:47.899617 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-02923bdf-c010-4373-b57b-7612a4002f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:47.999733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1109/2183] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:33:48.011278 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d2554-ea55-4600-851b-f8e49bfacf0c admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:48.064122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02923bdf-c010-4373-b57b-7612a4002f5a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:48.065829 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-02923bdf-c010-4373-b57b-7612a4002f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:48.087505 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02923bdf-c010-4373-b57b-7612a4002f5a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:48.095620 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0acef88-2476-4247-bc98-1440066b908e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:48.097110 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0acef88-2476-4247-bc98-1440066b908e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:48.097585 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0acef88-2476-4247-bc98-1440066b908e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:48.098052 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0acef88-2476-4247-bc98-1440066b908e admin admin] PATH_INFO: `/v3/projects/6a6a20b925f64e7b97e4076342753d35` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:48.100234 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0acef88-2476-4247-bc98-1440066b908e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a6a20b925f64e7b97e4076342753d35)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:48.103811 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0acef88-2476-4247-bc98-1440066b908e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:48.130040 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d2554-ea55-4600-851b-f8e49bfacf0c admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:48.135096 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3eb678-c002-48d4-aec8-4ce3c69248b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:48.136327 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3eb678-c002-48d4-aec8-4ce3c69248b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:48.136825 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3eb678-c002-48d4-aec8-4ce3c69248b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:48.137235 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3eb678-c002-48d4-aec8-4ce3c69248b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:48.151853 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3eb678-c002-48d4-aec8-4ce3c69248b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:48.154980 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3eb678-c002-48d4-aec8-4ce3c69248b0 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:48.171266 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1075/2184] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:33:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:33:48.173086 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d0acef88-2476-4247-bc98-1440066b908e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a6a20b925f64e7b97e4076342753d35', 'request_id': 'req-d0acef88-2476-4247-bc98-1440066b908e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:48.180933 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d0acef88-2476-4247-bc98-1440066b908e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a6a20b925f64e7b97e4076342753d35', 'request_id': 'req-d0acef88-2476-4247-bc98-1440066b908e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:48.189207 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d0acef88-2476-4247-bc98-1440066b908e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a6a20b925f64e7b97e4076342753d35', 'request_id': 'req-d0acef88-2476-4247-bc98-1440066b908e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:48.209240 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d0acef88-2476-4247-bc98-1440066b908e admin admin] The token cache is being invalidate because project 6a6a20b925f64e7b97e4076342753d35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:33:48.209563 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d0acef88-2476-4247-bc98-1440066b908e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:48.210469 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d0acef88-2476-4247-bc98-1440066b908e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:48.213410 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1110/2185] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:33:48 2022] DELETE /identity/v3/projects/6a6a20b925f64e7b97e4076342753d35 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:33:48.220306 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf2528-b7f1-4080-b164-be74e8b3bbb2 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:48.220729 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf2528-b7f1-4080-b164-be74e8b3bbb2 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:48.221303 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf2528-b7f1-4080-b164-be74e8b3bbb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:48.250383 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-4acf2528-b7f1-4080-b164-be74e8b3bbb2 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:48.252268 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4acf2528-b7f1-4080-b164-be74e8b3bbb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:48.340088 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:48.402200 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af51cbd-dfe8-45bd-9d3d-0f3bbcf979dc None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:48.403747 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af51cbd-dfe8-45bd-9d3d-0f3bbcf979dc None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:48.404129 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af51cbd-dfe8-45bd-9d3d-0f3bbcf979dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:48.407798 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1076/2186] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:48.438457 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-4af51cbd-dfe8-45bd-9d3d-0f3bbcf979dc None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:48.440233 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4af51cbd-dfe8-45bd-9d3d-0f3bbcf979dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:48.453403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1111/2187] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:48.454797 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf2528-b7f1-4080-b164-be74e8b3bbb2 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:48.465352 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf2528-b7f1-4080-b164-be74e8b3bbb2 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:48.471050 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0fb3b8-d25e-4133-87b0-ca86eaad4d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:48.471874 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0fb3b8-d25e-4133-87b0-ca86eaad4d32 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:48.472222 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0fb3b8-d25e-4133-87b0-ca86eaad4d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:48.472690 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0fb3b8-d25e-4133-87b0-ca86eaad4d32 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:48.473555 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0fb3b8-d25e-4133-87b0-ca86eaad4d32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:33:48.473859 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0fb3b8-d25e-4133-87b0-ca86eaad4d32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:48.475564 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0fb3b8-d25e-4133-87b0-ca86eaad4d32 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:48.493378 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1077/2188] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:33:48.494862 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af51cbd-dfe8-45bd-9d3d-0f3bbcf979dc None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:48.507642 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af51cbd-dfe8-45bd-9d3d-0f3bbcf979dc None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:48.513406 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6f160-9514-4e5b-b63c-d86bf7f7620b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:48.514103 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6f160-9514-4e5b-b63c-d86bf7f7620b admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:48.514422 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6f160-9514-4e5b-b63c-d86bf7f7620b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:48.514700 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6f160-9514-4e5b-b63c-d86bf7f7620b admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:48.515413 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d6f160-9514-4e5b-b63c-d86bf7f7620b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:48.517982 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d6f160-9514-4e5b-b63c-d86bf7f7620b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:48.560640 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1112/2189] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:33:48 2022] POST /identity/v3/projects => generated 379 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:33:48.562384 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0fb3b8-d25e-4133-87b0-ca86eaad4d32 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:48.592541 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0fb3b8-d25e-4133-87b0-ca86eaad4d32 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:48.693868 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3100580b-0a8a-440e-970f-dd234cfdd83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:48.695298 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3100580b-0a8a-440e-970f-dd234cfdd83e admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:48.695828 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3100580b-0a8a-440e-970f-dd234cfdd83e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:48.697037 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3100580b-0a8a-440e-970f-dd234cfdd83e admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:48.697815 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3100580b-0a8a-440e-970f-dd234cfdd83e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:48.702360 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3100580b-0a8a-440e-970f-dd234cfdd83e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:48.739552 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1078/2190] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:33:48 2022] POST /identity/v3/users => generated 290 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:33:48.745022 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6f160-9514-4e5b-b63c-d86bf7f7620b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:48.758435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6f160-9514-4e5b-b63c-d86bf7f7620b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:48.767412 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aae875-09df-485c-adfc-e98224da81f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:48.769035 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aae875-09df-485c-adfc-e98224da81f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:48.769535 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aae875-09df-485c-adfc-e98224da81f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:48.769990 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aae875-09df-485c-adfc-e98224da81f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:48.771460 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aae875-09df-485c-adfc-e98224da81f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:48.774668 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aae875-09df-485c-adfc-e98224da81f1 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:48.799926 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1113/2191] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:33:48.802085 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3100580b-0a8a-440e-970f-dd234cfdd83e admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:48.815026 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3100580b-0a8a-440e-970f-dd234cfdd83e admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:48.820430 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf696c-950f-4bd1-85ab-bb8a3f972c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:48.821198 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf696c-950f-4bd1-85ab-bb8a3f972c45 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:48.821414 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf696c-950f-4bd1-85ab-bb8a3f972c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:48.821696 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf696c-950f-4bd1-85ab-bb8a3f972c45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:48.822519 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaf696c-950f-4bd1-85ab-bb8a3f972c45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:48.824511 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaf696c-950f-4bd1-85ab-bb8a3f972c45 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:48.841636 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1079/2192] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:33:48.843701 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aae875-09df-485c-adfc-e98224da81f1 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:48.872202 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aae875-09df-485c-adfc-e98224da81f1 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:48.880303 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e746e-056d-4d9d-9845-34c57f922fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:48.881939 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e746e-056d-4d9d-9845-34c57f922fbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:48.882494 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e746e-056d-4d9d-9845-34c57f922fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:48.883030 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e746e-056d-4d9d-9845-34c57f922fbf admin admin] PATH_INFO: `/v3/projects/4951b93f86e1485692a4aae7b9ece8f6/users/0a58f82117ab437cab5607c204c86547/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:48.909690 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e746e-056d-4d9d-9845-34c57f922fbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=4951b93f86e1485692a4aae7b9ece8f6, user_id=0a58f82117ab437cab5607c204c86547, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:48.915148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e746e-056d-4d9d-9845-34c57f922fbf admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:48.940567 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1114/2193] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:33:48 2022] PUT /identity/v3/projects/4951b93f86e1485692a4aae7b9ece8f6/users/0a58f82117ab437cab5607c204c86547/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:33:48.947852 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea63cc-6e8e-4619-9414-01a57bdfe967 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:48.947852 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea63cc-6e8e-4619-9414-01a57bdfe967 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:48.947852 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea63cc-6e8e-4619-9414-01a57bdfe967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:48.976030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-12ea63cc-6e8e-4619-9414-01a57bdfe967 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:48.978358 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-12ea63cc-6e8e-4619-9414-01a57bdfe967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:49.127440 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1080/2194] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:33:49.140367 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e746e-056d-4d9d-9845-34c57f922fbf admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:49.144432 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-237e746e-056d-4d9d-9845-34c57f922fbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:49.149880 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b33f5-2c5e-4265-a668-b7eaf9b8c82e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:49.150494 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b33f5-2c5e-4265-a668-b7eaf9b8c82e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:49.150964 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b33f5-2c5e-4265-a668-b7eaf9b8c82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:49.177404 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:49.203045 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-d43b33f5-2c5e-4265-a668-b7eaf9b8c82e None None] MFA Rules not processed for user `c5dc7ff3a5624d8ca2cf116f8f9f0b2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:49.206363 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d43b33f5-2c5e-4265-a668-b7eaf9b8c82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:49.260459 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e746e-056d-4d9d-9845-34c57f922fbf admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:49.280063 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a679058-4b77-4f6f-90b2-45bb0623c187 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:49.280851 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a679058-4b77-4f6f-90b2-45bb0623c187 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:49.281194 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a679058-4b77-4f6f-90b2-45bb0623c187 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:49.281524 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a679058-4b77-4f6f-90b2-45bb0623c187 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:49.300153 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a679058-4b77-4f6f-90b2-45bb0623c187 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:49.302893 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a679058-4b77-4f6f-90b2-45bb0623c187 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:49.308660 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:49.331652 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1115/2195] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:33:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:33:49.359049 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1081/2196] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:49.365002 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a679058-4b77-4f6f-90b2-45bb0623c187 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:49.367567 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7a679058-4b77-4f6f-90b2-45bb0623c187 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:49.390546 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a679058-4b77-4f6f-90b2-45bb0623c187 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:49.397354 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df186180-ad4a-4d82-b9db-f545a83aac0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:49.398542 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df186180-ad4a-4d82-b9db-f545a83aac0b admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:49.399062 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df186180-ad4a-4d82-b9db-f545a83aac0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:49.399543 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df186180-ad4a-4d82-b9db-f545a83aac0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:49.400624 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df186180-ad4a-4d82-b9db-f545a83aac0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:49.403263 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df186180-ad4a-4d82-b9db-f545a83aac0b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:49.445829 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1116/2197] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:33:49 2022] POST /identity/v3/projects => generated 373 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:33:49.452309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b33f5-2c5e-4265-a668-b7eaf9b8c82e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:49.490678 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b33f5-2c5e-4265-a668-b7eaf9b8c82e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:49.609188 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a7815-3e3f-4ddc-b542-d900151abc30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:49.610376 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a7815-3e3f-4ddc-b542-d900151abc30 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:49.610840 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a7815-3e3f-4ddc-b542-d900151abc30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:49.611811 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a7815-3e3f-4ddc-b542-d900151abc30 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:49.612884 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5a7815-3e3f-4ddc-b542-d900151abc30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:49.615315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5a7815-3e3f-4ddc-b542-d900151abc30 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:49.648259 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1082/2198] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:33:49 2022] POST /identity/v3/users => generated 293 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:33:49.653682 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df186180-ad4a-4d82-b9db-f545a83aac0b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:49.666845 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df186180-ad4a-4d82-b9db-f545a83aac0b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:49.671947 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc32355-8ae8-46e3-b5b5-3fc8da7cbf64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:49.672733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc32355-8ae8-46e3-b5b5-3fc8da7cbf64 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:49.674037 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc32355-8ae8-46e3-b5b5-3fc8da7cbf64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:49.674037 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc32355-8ae8-46e3-b5b5-3fc8da7cbf64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:49.674413 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc32355-8ae8-46e3-b5b5-3fc8da7cbf64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:49.676194 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc32355-8ae8-46e3-b5b5-3fc8da7cbf64 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:49.693756 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1117/2199] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 21:33:49.699300 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a7815-3e3f-4ddc-b542-d900151abc30 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:49.713081 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a7815-3e3f-4ddc-b542-d900151abc30 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:49.722400 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa0b5dc-ce84-42b8-b6ca-0a4b67878f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:49.723960 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0b5dc-ce84-42b8-b6ca-0a4b67878f99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:49.724400 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0b5dc-ce84-42b8-b6ca-0a4b67878f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:49.724830 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0b5dc-ce84-42b8-b6ca-0a4b67878f99 admin admin] PATH_INFO: `/v3/projects/11f59c8c907e42049a28d8c5bb8eba97/users/d9820c3bc153466cadfcbaa2bb9233c9/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:49.749398 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa0b5dc-ce84-42b8-b6ca-0a4b67878f99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11f59c8c907e42049a28d8c5bb8eba97, user_id=d9820c3bc153466cadfcbaa2bb9233c9, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:49.753781 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa0b5dc-ce84-42b8-b6ca-0a4b67878f99 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:49.773926 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1083/2200] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:33:49 2022] PUT /identity/v3/projects/11f59c8c907e42049a28d8c5bb8eba97/users/d9820c3bc153466cadfcbaa2bb9233c9/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:49.779027 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc32355-8ae8-46e3-b5b5-3fc8da7cbf64 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:49.796018 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc32355-8ae8-46e3-b5b5-3fc8da7cbf64 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:49.922284 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ef28c-bd7e-442d-9313-b31d4044d4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:49.923803 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ef28c-bd7e-442d-9313-b31d4044d4ef admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:49.924108 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ef28c-bd7e-442d-9313-b31d4044d4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:49.924394 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ef28c-bd7e-442d-9313-b31d4044d4ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:49.925226 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ef28c-bd7e-442d-9313-b31d4044d4ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:49.927098 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ef28c-bd7e-442d-9313-b31d4044d4ef admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:49.949300 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1118/2201] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:33:49.950918 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa0b5dc-ce84-42b8-b6ca-0a4b67878f99 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:49.966206 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa0b5dc-ce84-42b8-b6ca-0a4b67878f99 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:49.972465 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d365bc6e-49b1-4ffc-9f3c-93170889a7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:49.973401 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d365bc6e-49b1-4ffc-9f3c-93170889a7e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:49.973688 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d365bc6e-49b1-4ffc-9f3c-93170889a7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:49.973957 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d365bc6e-49b1-4ffc-9f3c-93170889a7e5 admin admin] PATH_INFO: `/v3/projects/11f59c8c907e42049a28d8c5bb8eba97/users/d9820c3bc153466cadfcbaa2bb9233c9/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:49.977181 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d365bc6e-49b1-4ffc-9f3c-93170889a7e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11f59c8c907e42049a28d8c5bb8eba97, user_id=d9820c3bc153466cadfcbaa2bb9233c9, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:49.980034 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d365bc6e-49b1-4ffc-9f3c-93170889a7e5 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:49.994594 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1084/2202] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:33:49 2022] PUT /identity/v3/projects/11f59c8c907e42049a28d8c5bb8eba97/users/d9820c3bc153466cadfcbaa2bb9233c9/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:51.205819 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5c6c5-aa28-4ff4-8315-526ec4921e1e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:51.207642 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5c6c5-aa28-4ff4-8315-526ec4921e1e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:51.208272 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5c6c5-aa28-4ff4-8315-526ec4921e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:51.262783 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-4dd5c6c5-aa28-4ff4-8315-526ec4921e1e None None] MFA Rules not processed for user `d9820c3bc153466cadfcbaa2bb9233c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:51.264818 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4dd5c6c5-aa28-4ff4-8315-526ec4921e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:51.393428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1119/2203] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:51 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:51.411049 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d365bc6e-49b1-4ffc-9f3c-93170889a7e5 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:51.414026 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d365bc6e-49b1-4ffc-9f3c-93170889a7e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:51.561792 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d365bc6e-49b1-4ffc-9f3c-93170889a7e5 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:51.579236 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2ea15-72b4-40a3-9f01-432e17624586 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:51.579949 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2ea15-72b4-40a3-9f01-432e17624586 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:51.580211 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2ea15-72b4-40a3-9f01-432e17624586 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:51.580492 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2ea15-72b4-40a3-9f01-432e17624586 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:51.715095 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:51.751500 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2ea15-72b4-40a3-9f01-432e17624586 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:51.753335 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2ea15-72b4-40a3-9f01-432e17624586 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:51.774088 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1085/2204] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:33:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:33:54.248903 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49476ced-b747-4408-9f73-1deeb0adecc1 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:54.250051 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49476ced-b747-4408-9f73-1deeb0adecc1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:54.252675 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49476ced-b747-4408-9f73-1deeb0adecc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:54.279272 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-49476ced-b747-4408-9f73-1deeb0adecc1 None None] MFA Rules not processed for user `47575a76d62b4a109fda03f4bf016fc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:54.281827 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-49476ced-b747-4408-9f73-1deeb0adecc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:54.291940 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:54.404205 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1120/2205] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:33:55.485129 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2ea15-72b4-40a3-9f01-432e17624586 service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:55.498420 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e1f2ea15-72b4-40a3-9f01-432e17624586 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:55.790275 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49476ced-b747-4408-9f73-1deeb0adecc1 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:55.801665 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2ea15-72b4-40a3-9f01-432e17624586 service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:55.805249 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49476ced-b747-4408-9f73-1deeb0adecc1 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:55.812485 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4338e1-1623-4de7-98b6-b299027999dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:55.814192 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4338e1-1623-4de7-98b6-b299027999dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:55.814518 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4338e1-1623-4de7-98b6-b299027999dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:55.814790 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4338e1-1623-4de7-98b6-b299027999dc admin admin] PATH_INFO: `/v3/users/997d7aa08d444370a63e06b286d9fa66` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:55.817966 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4338e1-1623-4de7-98b6-b299027999dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=997d7aa08d444370a63e06b286d9fa66)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:55.819332 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744c707a-fc5c-45e6-9ac2-fd2002d340db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:55.820235 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4338e1-1623-4de7-98b6-b299027999dc admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:55.824472 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c707a-fc5c-45e6-9ac2-fd2002d340db service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:55.825313 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c707a-fc5c-45e6-9ac2-fd2002d340db service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:55.826913 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c707a-fc5c-45e6-9ac2-fd2002d340db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:55.875801 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744c707a-fc5c-45e6-9ac2-fd2002d340db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:55.884396 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744c707a-fc5c-45e6-9ac2-fd2002d340db service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:55.922662 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1121/2206] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:33:55 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:33:55.945129 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4d4338e1-1623-4de7-98b6-b299027999dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '997d7aa08d444370a63e06b286d9fa66'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:55.945905 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4d4338e1-1623-4de7-98b6-b299027999dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '997d7aa08d444370a63e06b286d9fa66'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:56.008350 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4d4338e1-1623-4de7-98b6-b299027999dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '997d7aa08d444370a63e06b286d9fa66'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:56.069483 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4d4338e1-1623-4de7-98b6-b299027999dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '997d7aa08d444370a63e06b286d9fa66'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:56.097969 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4d4338e1-1623-4de7-98b6-b299027999dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '997d7aa08d444370a63e06b286d9fa66'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:56.118138 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1086/2207] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:33:55 2022] DELETE /identity/v3/users/997d7aa08d444370a63e06b286d9fa66 => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:33:56.122886 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0612e-354a-41c3-b081-f5b12d8de5d3 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:56.123571 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0612e-354a-41c3-b081-f5b12d8de5d3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:56.124041 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0612e-354a-41c3-b081-f5b12d8de5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:56.152318 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a3f0612e-354a-41c3-b081-f5b12d8de5d3 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:56.161865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a3f0612e-354a-41c3-b081-f5b12d8de5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:56.488671 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1122/2208] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:33:56.504641 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4338e1-1623-4de7-98b6-b299027999dc admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:56.507685 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4d4338e1-1623-4de7-98b6-b299027999dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:56.694473 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4338e1-1623-4de7-98b6-b299027999dc admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:56.702334 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e930ab2-f1e3-4187-b328-f63366af136e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:56.704373 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e930ab2-f1e3-4187-b328-f63366af136e service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:56.704905 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e930ab2-f1e3-4187-b328-f63366af136e service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:56.705410 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e930ab2-f1e3-4187-b328-f63366af136e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:56.724788 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e930ab2-f1e3-4187-b328-f63366af136e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:56.729376 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e930ab2-f1e3-4187-b328-f63366af136e service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:56.755487 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1087/2209] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:33:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:33:57.968210 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f0612e-354a-41c3-b081-f5b12d8de5d3 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:57.971706 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a3f0612e-354a-41c3-b081-f5b12d8de5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:58.017379 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f0612e-354a-41c3-b081-f5b12d8de5d3 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:58.036546 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a602c2c-27a1-43ca-beb4-af43bad77124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:58.037975 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a602c2c-27a1-43ca-beb4-af43bad77124 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:58.039685 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a602c2c-27a1-43ca-beb4-af43bad77124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:58.040257 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a602c2c-27a1-43ca-beb4-af43bad77124 admin admin] PATH_INFO: `/v3/projects/a69c8ac8a7d34c0fa6b1632a0c15ae7d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:58.043042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a602c2c-27a1-43ca-beb4-af43bad77124 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a69c8ac8a7d34c0fa6b1632a0c15ae7d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:58.046287 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a602c2c-27a1-43ca-beb4-af43bad77124 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:58.253698 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4a602c2c-27a1-43ca-beb4-af43bad77124 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a69c8ac8a7d34c0fa6b1632a0c15ae7d', 'request_id': 'req-4a602c2c-27a1-43ca-beb4-af43bad77124'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:58.279569 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4a602c2c-27a1-43ca-beb4-af43bad77124 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a69c8ac8a7d34c0fa6b1632a0c15ae7d', 'request_id': 'req-4a602c2c-27a1-43ca-beb4-af43bad77124'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:58.292810 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4a602c2c-27a1-43ca-beb4-af43bad77124 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a69c8ac8a7d34c0fa6b1632a0c15ae7d', 'request_id': 'req-4a602c2c-27a1-43ca-beb4-af43bad77124'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:58.350038 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facef2de-ab2f-450d-9962-2139d4c99d3c None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:58.350979 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facef2de-ab2f-450d-9962-2139d4c99d3c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:58.351146 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facef2de-ab2f-450d-9962-2139d4c99d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:58.362444 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4a602c2c-27a1-43ca-beb4-af43bad77124 admin admin] The token cache is being invalidate because project a69c8ac8a7d34c0fa6b1632a0c15ae7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:33:58.363105 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4a602c2c-27a1-43ca-beb4-af43bad77124 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:58.366504 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4a602c2c-27a1-43ca-beb4-af43bad77124 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:58.387392 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1123/2210] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:33:57 2022] DELETE /identity/v3/projects/a69c8ac8a7d34c0fa6b1632a0c15ae7d => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:58.407031 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a602c2c-27a1-43ca-beb4-af43bad77124 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:58.413233 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4a602c2c-27a1-43ca-beb4-af43bad77124 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:58.458153 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:58.534389 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-facef2de-ab2f-450d-9962-2139d4c99d3c None None] MFA Rules not processed for user `0a58f82117ab437cab5607c204c86547`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:33:58.536156 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-facef2de-ab2f-450d-9962-2139d4c99d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:58.537279 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:58.678725 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a602c2c-27a1-43ca-beb4-af43bad77124 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:58.688762 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c18ef2-cb1e-4d26-9852-c293e31da50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:58.690595 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c18ef2-cb1e-4d26-9852-c293e31da50b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:58.691151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c18ef2-cb1e-4d26-9852-c293e31da50b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:58.691640 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c18ef2-cb1e-4d26-9852-c293e31da50b admin admin] PATH_INFO: `/v3/users/be4d35be6a2b48b5a750ef53b6490933` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:58.694751 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c18ef2-cb1e-4d26-9852-c293e31da50b admin admin] RBAC: Authorizing `identity:delete_user(user_id=be4d35be6a2b48b5a750ef53b6490933)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:58.698493 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c18ef2-cb1e-4d26-9852-c293e31da50b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:58.746476 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1088/2211] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:33:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 399 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:33:58.750756 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facef2de-ab2f-450d-9962-2139d4c99d3c None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:58.757704 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-facef2de-ab2f-450d-9962-2139d4c99d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:58.805825 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e6c18ef2-cb1e-4d26-9852-c293e31da50b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be4d35be6a2b48b5a750ef53b6490933'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:58.819981 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facef2de-ab2f-450d-9962-2139d4c99d3c None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:58.830127 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e29a5-4c62-4b58-a5ec-41b895e30416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:58.831704 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e29a5-4c62-4b58-a5ec-41b895e30416 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:58.832479 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e29a5-4c62-4b58-a5ec-41b895e30416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:58.833144 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e29a5-4c62-4b58-a5ec-41b895e30416 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:58.838423 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364e29a5-4c62-4b58-a5ec-41b895e30416 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:58.838864 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e6c18ef2-cb1e-4d26-9852-c293e31da50b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be4d35be6a2b48b5a750ef53b6490933'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:58.839599 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e6c18ef2-cb1e-4d26-9852-c293e31da50b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be4d35be6a2b48b5a750ef53b6490933'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:58.840482 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e6c18ef2-cb1e-4d26-9852-c293e31da50b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be4d35be6a2b48b5a750ef53b6490933'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:58.843782 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364e29a5-4c62-4b58-a5ec-41b895e30416 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:58.863691 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e6c18ef2-cb1e-4d26-9852-c293e31da50b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be4d35be6a2b48b5a750ef53b6490933'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:33:58.893853 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1124/2212] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:33:58 2022] DELETE /identity/v3/users/be4d35be6a2b48b5a750ef53b6490933 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:58.894599 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1089/2213] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:33:58 2022] POST /identity/v3/projects => generated 377 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:33:58.895878 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c18ef2-cb1e-4d26-9852-c293e31da50b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:58.897466 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e6c18ef2-cb1e-4d26-9852-c293e31da50b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:59.037750 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c18ef2-cb1e-4d26-9852-c293e31da50b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:59.161527 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:33:59.182396 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea53942-3a7b-4d21-8740-5d338c593648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:59.183281 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea53942-3a7b-4d21-8740-5d338c593648 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:59.183281 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea53942-3a7b-4d21-8740-5d338c593648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:59.183599 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea53942-3a7b-4d21-8740-5d338c593648 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:59.187455 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea53942-3a7b-4d21-8740-5d338c593648 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:59.192588 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea53942-3a7b-4d21-8740-5d338c593648 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:59.242811 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1125/2214] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:33:58 2022] POST /identity/v3/users => generated 295 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:33:59.244436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e29a5-4c62-4b58-a5ec-41b895e30416 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:59.260222 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e29a5-4c62-4b58-a5ec-41b895e30416 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:59.273729 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f216dcc0-f914-4a0c-ab25-96e08fac8c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:59.274455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f216dcc0-f914-4a0c-ab25-96e08fac8c7f admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:59.274746 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f216dcc0-f914-4a0c-ab25-96e08fac8c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:59.275009 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f216dcc0-f914-4a0c-ab25-96e08fac8c7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:59.275917 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f216dcc0-f914-4a0c-ab25-96e08fac8c7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:59.277600 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f216dcc0-f914-4a0c-ab25-96e08fac8c7f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:59.307919 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1090/2215] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:59 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 21:33:59.312331 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea53942-3a7b-4d21-8740-5d338c593648 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:59.358714 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea53942-3a7b-4d21-8740-5d338c593648 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:59.381597 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99738b-88be-4476-bf09-a0d3723f65ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:59.383320 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99738b-88be-4476-bf09-a0d3723f65ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:59.383820 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99738b-88be-4476-bf09-a0d3723f65ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:59.384330 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99738b-88be-4476-bf09-a0d3723f65ad admin admin] PATH_INFO: `/v3/projects/8b2c1a02d52a4033956d358959ba1e6f/users/a991019b81a843d38987f889fec0eec9/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:59.440568 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a99738b-88be-4476-bf09-a0d3723f65ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b2c1a02d52a4033956d358959ba1e6f, user_id=a991019b81a843d38987f889fec0eec9, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:59.448354 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a99738b-88be-4476-bf09-a0d3723f65ad admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:59.483104 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1126/2216] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:33:59 2022] PUT /identity/v3/projects/8b2c1a02d52a4033956d358959ba1e6f/users/a991019b81a843d38987f889fec0eec9/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:59.488503 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f216dcc0-f914-4a0c-ab25-96e08fac8c7f admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:59.536223 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f216dcc0-f914-4a0c-ab25-96e08fac8c7f admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:59.727888 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04859b27-0c4d-4b4b-ae9b-ab392f8c95da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:59.730695 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04859b27-0c4d-4b4b-ae9b-ab392f8c95da admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:59.730965 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04859b27-0c4d-4b4b-ae9b-ab392f8c95da admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:59.731426 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04859b27-0c4d-4b4b-ae9b-ab392f8c95da admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:59.732543 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04859b27-0c4d-4b4b-ae9b-ab392f8c95da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:59.739785 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04859b27-0c4d-4b4b-ae9b-ab392f8c95da admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:59.784231 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1091/2217] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:33:59 2022] GET /identity/v3/roles => generated 1420 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:33:59.785321 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99738b-88be-4476-bf09-a0d3723f65ad admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:59.801027 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99738b-88be-4476-bf09-a0d3723f65ad admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:33:59.808542 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a977fc86-ee90-4005-9494-0e574d55ed66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:33:59.810074 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a977fc86-ee90-4005-9494-0e574d55ed66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:33:59.810670 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a977fc86-ee90-4005-9494-0e574d55ed66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:33:59.811079 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a977fc86-ee90-4005-9494-0e574d55ed66 admin admin] PATH_INFO: `/v3/projects/8b2c1a02d52a4033956d358959ba1e6f/users/a991019b81a843d38987f889fec0eec9/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:33:59.814509 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a977fc86-ee90-4005-9494-0e574d55ed66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b2c1a02d52a4033956d358959ba1e6f, user_id=a991019b81a843d38987f889fec0eec9, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:33:59.818309 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a977fc86-ee90-4005-9494-0e574d55ed66 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:33:59.841592 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1127/2218] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:33:59 2022] PUT /identity/v3/projects/8b2c1a02d52a4033956d358959ba1e6f/users/a991019b81a843d38987f889fec0eec9/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:33:59.957537 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04859b27-0c4d-4b4b-ae9b-ab392f8c95da admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:33:59.962966 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-04859b27-0c4d-4b4b-ae9b-ab392f8c95da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:33:59.992443 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04859b27-0c4d-4b4b-ae9b-ab392f8c95da admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:00.182750 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09900259-ae50-445d-8533-b61d58efbaf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:00.182750 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09900259-ae50-445d-8533-b61d58efbaf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:00.182750 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09900259-ae50-445d-8533-b61d58efbaf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:00.182750 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09900259-ae50-445d-8533-b61d58efbaf6 admin admin] PATH_INFO: `/v3/projects/d6fbdeaef61748ebbe25e365ec81f561` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:00.188880 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09900259-ae50-445d-8533-b61d58efbaf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6fbdeaef61748ebbe25e365ec81f561)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:00.193457 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09900259-ae50-445d-8533-b61d58efbaf6 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:00.198369 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:34:00.336530 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-09900259-ae50-445d-8533-b61d58efbaf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6fbdeaef61748ebbe25e365ec81f561', 'request_id': 'req-09900259-ae50-445d-8533-b61d58efbaf6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:00.357168 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-09900259-ae50-445d-8533-b61d58efbaf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6fbdeaef61748ebbe25e365ec81f561', 'request_id': 'req-09900259-ae50-445d-8533-b61d58efbaf6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:00.390372 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-09900259-ae50-445d-8533-b61d58efbaf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6fbdeaef61748ebbe25e365ec81f561', 'request_id': 'req-09900259-ae50-445d-8533-b61d58efbaf6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:00.438581 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-09900259-ae50-445d-8533-b61d58efbaf6 admin admin] The token cache is being invalidate because project d6fbdeaef61748ebbe25e365ec81f561 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:34:00.439098 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-09900259-ae50-445d-8533-b61d58efbaf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:00.440324 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-09900259-ae50-445d-8533-b61d58efbaf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:00.446304 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1092/2219] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:33:59 2022] DELETE /identity/v3/projects/d6fbdeaef61748ebbe25e365ec81f561 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:34:00.449522 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f3e29-3632-4d65-9b61-2abd7e7a30cd None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:00.450429 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f3e29-3632-4d65-9b61-2abd7e7a30cd None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:00.450429 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f3e29-3632-4d65-9b61-2abd7e7a30cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:00.540588 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-646f3e29-3632-4d65-9b61-2abd7e7a30cd None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:34:00.541873 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-646f3e29-3632-4d65-9b61-2abd7e7a30cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:00.711973 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1128/2220] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:34:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:34:00.721683 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc7436-71ae-44f3-9047-02d24d038622 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:00.723463 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc7436-71ae-44f3-9047-02d24d038622 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:00.724167 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc7436-71ae-44f3-9047-02d24d038622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:00.793222 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b0cc7436-71ae-44f3-9047-02d24d038622 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:34:00.800717 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b0cc7436-71ae-44f3-9047-02d24d038622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:00.848825 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1093/2221] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:34:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:34:00.851796 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f3e29-3632-4d65-9b61-2abd7e7a30cd None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:00.888695 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f3e29-3632-4d65-9b61-2abd7e7a30cd None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:00.897950 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce53e21-0b6f-4235-a79f-3bf0fb05dbf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:00.899128 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce53e21-0b6f-4235-a79f-3bf0fb05dbf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:00.899580 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce53e21-0b6f-4235-a79f-3bf0fb05dbf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:00.900410 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce53e21-0b6f-4235-a79f-3bf0fb05dbf9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:00.901765 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce53e21-0b6f-4235-a79f-3bf0fb05dbf9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:34:00.902342 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce53e21-0b6f-4235-a79f-3bf0fb05dbf9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:00.905772 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce53e21-0b6f-4235-a79f-3bf0fb05dbf9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:00.986940 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1129/2222] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:34:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:34:00.993413 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc7436-71ae-44f3-9047-02d24d038622 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:01.034685 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc7436-71ae-44f3-9047-02d24d038622 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:01.051142 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec7db10-d8fd-4bb6-a036-bd6f4f6505fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:01.051856 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7db10-d8fd-4bb6-a036-bd6f4f6505fa admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:01.052142 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7db10-d8fd-4bb6-a036-bd6f4f6505fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:01.052421 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7db10-d8fd-4bb6-a036-bd6f4f6505fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:01.053147 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec7db10-d8fd-4bb6-a036-bd6f4f6505fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:01.063003 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec7db10-d8fd-4bb6-a036-bd6f4f6505fa admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:01.169020 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1094/2223] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:34:00 2022] POST /identity/v3/projects => generated 377 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:34:01.172198 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce53e21-0b6f-4235-a79f-3bf0fb05dbf9 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:01.221005 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce53e21-0b6f-4235-a79f-3bf0fb05dbf9 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:01.499487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39bd852-416b-4dc2-8df3-19f2e9b984a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:01.501959 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39bd852-416b-4dc2-8df3-19f2e9b984a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:01.505369 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39bd852-416b-4dc2-8df3-19f2e9b984a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:01.505369 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39bd852-416b-4dc2-8df3-19f2e9b984a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:01.512187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39bd852-416b-4dc2-8df3-19f2e9b984a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:01.518709 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39bd852-416b-4dc2-8df3-19f2e9b984a8 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:01.577027 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1130/2224] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:34:01 2022] POST /identity/v3/users => generated 289 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:34:01.578685 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec7db10-d8fd-4bb6-a036-bd6f4f6505fa admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:01.615384 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec7db10-d8fd-4bb6-a036-bd6f4f6505fa admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:01.640778 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd4686-53ec-4095-89e8-5bcdca6ba18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:01.644526 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd4686-53ec-4095-89e8-5bcdca6ba18a admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:01.645002 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd4686-53ec-4095-89e8-5bcdca6ba18a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:01.648173 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd4686-53ec-4095-89e8-5bcdca6ba18a admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:01.649801 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cd4686-53ec-4095-89e8-5bcdca6ba18a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:01.652919 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cd4686-53ec-4095-89e8-5bcdca6ba18a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:01.695882 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1095/2225] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:34:01 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:34:01.697712 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39bd852-416b-4dc2-8df3-19f2e9b984a8 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:01.713498 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39bd852-416b-4dc2-8df3-19f2e9b984a8 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:01.729956 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f43a5-a041-4d2e-a007-4b764a5a2fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:01.731247 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f43a5-a041-4d2e-a007-4b764a5a2fbf admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:01.731662 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f43a5-a041-4d2e-a007-4b764a5a2fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:01.732023 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f43a5-a041-4d2e-a007-4b764a5a2fbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:01.733561 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76f43a5-a041-4d2e-a007-4b764a5a2fbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:01.735993 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76f43a5-a041-4d2e-a007-4b764a5a2fbf admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:01.771229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1131/2226] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:34:01 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:34:01.775387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd4686-53ec-4095-89e8-5bcdca6ba18a admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:01.820115 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd4686-53ec-4095-89e8-5bcdca6ba18a admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:01.831848 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe3b9e-d624-454b-861f-f07e1f71d191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:01.835615 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe3b9e-d624-454b-861f-f07e1f71d191 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:01.839869 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe3b9e-d624-454b-861f-f07e1f71d191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:01.840763 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe3b9e-d624-454b-861f-f07e1f71d191 admin admin] PATH_INFO: `/v3/projects/86b9e670ae6144548076173273026d17/users/591c0bf1ed2c401dbfd367dfff628b5f/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:01.911717 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abe3b9e-d624-454b-861f-f07e1f71d191 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86b9e670ae6144548076173273026d17, user_id=591c0bf1ed2c401dbfd367dfff628b5f, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:01.917796 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abe3b9e-d624-454b-861f-f07e1f71d191 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:01.960154 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1096/2227] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:34:01 2022] PUT /identity/v3/projects/86b9e670ae6144548076173273026d17/users/591c0bf1ed2c401dbfd367dfff628b5f/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:34:01.971217 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8149f7-ef62-4e54-8699-2ebf0f40066e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:01.971602 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8149f7-ef62-4e54-8699-2ebf0f40066e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:01.971964 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8149f7-ef62-4e54-8699-2ebf0f40066e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:01.981852 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:34:02.044603 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0f8149f7-ef62-4e54-8699-2ebf0f40066e None None] MFA Rules not processed for user `591c0bf1ed2c401dbfd367dfff628b5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:34:02.046548 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0f8149f7-ef62-4e54-8699-2ebf0f40066e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:02.219895 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1132/2228] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:34:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:34:02.238349 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe3b9e-d624-454b-861f-f07e1f71d191 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:02.240274 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3abe3b9e-d624-454b-861f-f07e1f71d191 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:02.343061 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe3b9e-d624-454b-861f-f07e1f71d191 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:02.348187 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a562e9-98e8-4760-875c-1bfc1a2ee64f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:02.348876 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a562e9-98e8-4760-875c-1bfc1a2ee64f service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:02.349159 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a562e9-98e8-4760-875c-1bfc1a2ee64f service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:02.349498 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a562e9-98e8-4760-875c-1bfc1a2ee64f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:02.361189 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a562e9-98e8-4760-875c-1bfc1a2ee64f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:02.363011 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a562e9-98e8-4760-875c-1bfc1a2ee64f service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:02.387684 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1097/2229] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:34:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:34:08.726714 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd597956-4ece-4226-a32d-83923764f0d7 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:08.727250 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd597956-4ece-4226-a32d-83923764f0d7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:08.727580 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd597956-4ece-4226-a32d-83923764f0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:08.769023 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-cd597956-4ece-4226-a32d-83923764f0d7 None None] MFA Rules not processed for user `a991019b81a843d38987f889fec0eec9`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:34:08.770923 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cd597956-4ece-4226-a32d-83923764f0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:08.881328 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1133/2230] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:34:08 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:34:08.895652 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a562e9-98e8-4760-875c-1bfc1a2ee64f service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:08.897181 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a9a562e9-98e8-4760-875c-1bfc1a2ee64f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:08.984837 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:34:09.048263 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a562e9-98e8-4760-875c-1bfc1a2ee64f service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:09.053872 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40649dc3-88e9-4621-82b8-92a3739a2e8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:09.054600 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40649dc3-88e9-4621-82b8-92a3739a2e8b service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:09.054897 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40649dc3-88e9-4621-82b8-92a3739a2e8b service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:09.055147 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40649dc3-88e9-4621-82b8-92a3739a2e8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:09.056991 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-40649dc3-88e9-4621-82b8-92a3739a2e8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:09.136663 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40649dc3-88e9-4621-82b8-92a3739a2e8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:09.139022 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40649dc3-88e9-4621-82b8-92a3739a2e8b service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:09.154336 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1098/2231] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:34:08 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:34:18.876847 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=65 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:34:19.148050 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=42 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:34:27.016411 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd597956-4ece-4226-a32d-83923764f0d7 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:27.018279 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cd597956-4ece-4226-a32d-83923764f0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:27.152183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd597956-4ece-4226-a32d-83923764f0d7 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:27.160436 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abae734-e3c5-44c4-8552-20bb78bfe8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:27.162096 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abae734-e3c5-44c4-8552-20bb78bfe8e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:27.162839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abae734-e3c5-44c4-8552-20bb78bfe8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:27.163131 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abae734-e3c5-44c4-8552-20bb78bfe8e1 admin admin] PATH_INFO: `/v3/users/75a97d18548d4ec48f07fc2bca608028` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:27.166405 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abae734-e3c5-44c4-8552-20bb78bfe8e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75a97d18548d4ec48f07fc2bca608028)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:27.168238 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abae734-e3c5-44c4-8552-20bb78bfe8e1 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:27.259853 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2abae734-e3c5-44c4-8552-20bb78bfe8e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75a97d18548d4ec48f07fc2bca608028'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:27.274954 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2abae734-e3c5-44c4-8552-20bb78bfe8e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75a97d18548d4ec48f07fc2bca608028'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:27.276469 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2abae734-e3c5-44c4-8552-20bb78bfe8e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75a97d18548d4ec48f07fc2bca608028'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:27.277686 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2abae734-e3c5-44c4-8552-20bb78bfe8e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75a97d18548d4ec48f07fc2bca608028'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:27.291487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2abae734-e3c5-44c4-8552-20bb78bfe8e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75a97d18548d4ec48f07fc2bca608028'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:27.314357 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1134/2232] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:34:27 2022] DELETE /identity/v3/users/75a97d18548d4ec48f07fc2bca608028 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:34:27.321162 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0efeaf-2a05-4bc9-8ef9-62f1d1d3e984 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:27.321770 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0efeaf-2a05-4bc9-8ef9-62f1d1d3e984 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:27.322035 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0efeaf-2a05-4bc9-8ef9-62f1d1d3e984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:27.387119 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-3b0efeaf-2a05-4bc9-8ef9-62f1d1d3e984 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:34:27.391577 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3b0efeaf-2a05-4bc9-8ef9-62f1d1d3e984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:27.472708 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1099/2233] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:34:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:34:27.484228 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abae734-e3c5-44c4-8552-20bb78bfe8e1 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:27.488697 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2abae734-e3c5-44c4-8552-20bb78bfe8e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:27.587951 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abae734-e3c5-44c4-8552-20bb78bfe8e1 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:27.600647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85caa2-ccf4-4d49-8299-2680a9d30c63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:27.601328 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85caa2-ccf4-4d49-8299-2680a9d30c63 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:27.601610 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85caa2-ccf4-4d49-8299-2680a9d30c63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:27.601883 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85caa2-ccf4-4d49-8299-2680a9d30c63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:27.621542 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff85caa2-ccf4-4d49-8299-2680a9d30c63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:27.622737 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff85caa2-ccf4-4d49-8299-2680a9d30c63 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:27.656530 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1135/2234] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:34:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:34:28.075647 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0efeaf-2a05-4bc9-8ef9-62f1d1d3e984 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:28.078229 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3b0efeaf-2a05-4bc9-8ef9-62f1d1d3e984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:28.097548 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0efeaf-2a05-4bc9-8ef9-62f1d1d3e984 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:28.104122 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10e5885-2503-420c-9ed8-49726811ef11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:28.105394 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10e5885-2503-420c-9ed8-49726811ef11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:28.105833 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10e5885-2503-420c-9ed8-49726811ef11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:28.106365 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10e5885-2503-420c-9ed8-49726811ef11 admin admin] PATH_INFO: `/v3/projects/979473217c604b749eb3ed2922e24939` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:28.108000 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10e5885-2503-420c-9ed8-49726811ef11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=979473217c604b749eb3ed2922e24939)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:28.110690 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10e5885-2503-420c-9ed8-49726811ef11 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:28.185583 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a10e5885-2503-420c-9ed8-49726811ef11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '979473217c604b749eb3ed2922e24939', 'request_id': 'req-a10e5885-2503-420c-9ed8-49726811ef11'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:28.193063 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a10e5885-2503-420c-9ed8-49726811ef11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '979473217c604b749eb3ed2922e24939', 'request_id': 'req-a10e5885-2503-420c-9ed8-49726811ef11'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:28.207929 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a10e5885-2503-420c-9ed8-49726811ef11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '979473217c604b749eb3ed2922e24939', 'request_id': 'req-a10e5885-2503-420c-9ed8-49726811ef11'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:28.224735 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a10e5885-2503-420c-9ed8-49726811ef11 admin admin] The token cache is being invalidate because project 979473217c604b749eb3ed2922e24939 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:34:28.225103 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a10e5885-2503-420c-9ed8-49726811ef11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:28.225979 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a10e5885-2503-420c-9ed8-49726811ef11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:28.227670 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1100/2235] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:34:28 2022] DELETE /identity/v3/projects/979473217c604b749eb3ed2922e24939 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:34:28.234165 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a98a5-818a-4e12-9170-cd7724ad918a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:28.235184 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a98a5-818a-4e12-9170-cd7724ad918a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:28.235900 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a98a5-818a-4e12-9170-cd7724ad918a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:28.264213 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-2b3a98a5-818a-4e12-9170-cd7724ad918a None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:34:28.266874 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2b3a98a5-818a-4e12-9170-cd7724ad918a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:28.292257 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:34:28.345943 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1136/2236] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:34:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:34:28.355018 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eeafb7-9e55-42bf-8a1b-c6aa8b3141ce None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:28.355850 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eeafb7-9e55-42bf-8a1b-c6aa8b3141ce None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:28.356003 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eeafb7-9e55-42bf-8a1b-c6aa8b3141ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:28.378867 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-57eeafb7-9e55-42bf-8a1b-c6aa8b3141ce None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:34:28.380334 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-57eeafb7-9e55-42bf-8a1b-c6aa8b3141ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:28.390702 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1101/2237] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:34:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:34:28.398306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3a98a5-818a-4e12-9170-cd7724ad918a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:28.412253 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3a98a5-818a-4e12-9170-cd7724ad918a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:28.424758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c566f0-5bfc-4220-8aa7-78e689e1e5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:28.425866 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c566f0-5bfc-4220-8aa7-78e689e1e5c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:28.426467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c566f0-5bfc-4220-8aa7-78e689e1e5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:28.426992 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c566f0-5bfc-4220-8aa7-78e689e1e5c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:28.428381 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c566f0-5bfc-4220-8aa7-78e689e1e5c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:34:28.428996 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c566f0-5bfc-4220-8aa7-78e689e1e5c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:28.432435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c566f0-5bfc-4220-8aa7-78e689e1e5c9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:28.452452 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1137/2238] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:34:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:34:28.456266 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eeafb7-9e55-42bf-8a1b-c6aa8b3141ce None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:28.466700 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eeafb7-9e55-42bf-8a1b-c6aa8b3141ce None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:28.473114 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52697d96-2a9b-44c0-a45b-405bd2f2437b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:28.473986 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52697d96-2a9b-44c0-a45b-405bd2f2437b admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:28.474199 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52697d96-2a9b-44c0-a45b-405bd2f2437b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:28.474538 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52697d96-2a9b-44c0-a45b-405bd2f2437b admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:28.475342 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52697d96-2a9b-44c0-a45b-405bd2f2437b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:28.477437 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52697d96-2a9b-44c0-a45b-405bd2f2437b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:28.511386 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1102/2239] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:34:28 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:34:28.517941 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c566f0-5bfc-4220-8aa7-78e689e1e5c9 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:28.531866 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c566f0-5bfc-4220-8aa7-78e689e1e5c9 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:28.609904 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea651c-e87a-4c64-b871-df72c4eb46ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:28.610784 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea651c-e87a-4c64-b871-df72c4eb46ca admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:28.611093 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea651c-e87a-4c64-b871-df72c4eb46ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:28.611379 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea651c-e87a-4c64-b871-df72c4eb46ca admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:28.612130 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcea651c-e87a-4c64-b871-df72c4eb46ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:28.614310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcea651c-e87a-4c64-b871-df72c4eb46ca admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:28.642751 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1138/2240] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:34:28 2022] POST /identity/v3/users => generated 287 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:34:28.647302 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52697d96-2a9b-44c0-a45b-405bd2f2437b admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:28.667822 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52697d96-2a9b-44c0-a45b-405bd2f2437b admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:28.676718 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35edb797-2741-4694-b7cb-cf9fb73bf757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:28.677612 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edb797-2741-4694-b7cb-cf9fb73bf757 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:28.677969 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edb797-2741-4694-b7cb-cf9fb73bf757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:28.678277 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edb797-2741-4694-b7cb-cf9fb73bf757 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:28.679671 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35edb797-2741-4694-b7cb-cf9fb73bf757 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:28.682793 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35edb797-2741-4694-b7cb-cf9fb73bf757 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:28.709107 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1103/2241] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:34:28 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 21:34:28.711702 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea651c-e87a-4c64-b871-df72c4eb46ca admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:28.732758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea651c-e87a-4c64-b871-df72c4eb46ca admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:28.740907 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa8ba49-a4e9-4d43-872c-9b5a3ed8951d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:28.742025 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8ba49-a4e9-4d43-872c-9b5a3ed8951d admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:28.742559 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8ba49-a4e9-4d43-872c-9b5a3ed8951d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:28.743044 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8ba49-a4e9-4d43-872c-9b5a3ed8951d admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:28.744417 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa8ba49-a4e9-4d43-872c-9b5a3ed8951d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:28.748190 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa8ba49-a4e9-4d43-872c-9b5a3ed8951d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:28.766280 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1139/2242] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:34:28 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:34:28.769109 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35edb797-2741-4694-b7cb-cf9fb73bf757 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:28.780564 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35edb797-2741-4694-b7cb-cf9fb73bf757 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:28.789410 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada359c7-306d-43cf-9631-c53c1da90d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:28.790463 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada359c7-306d-43cf-9631-c53c1da90d73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:28.790876 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada359c7-306d-43cf-9631-c53c1da90d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:28.791253 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada359c7-306d-43cf-9631-c53c1da90d73 admin admin] PATH_INFO: `/v3/projects/1d4148b35eac43a0bca89c631559ddfe/users/f1480d45394b4ae294d0a4b1b59af0d1/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:28.811183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada359c7-306d-43cf-9631-c53c1da90d73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d4148b35eac43a0bca89c631559ddfe, user_id=f1480d45394b4ae294d0a4b1b59af0d1, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:28.814633 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada359c7-306d-43cf-9631-c53c1da90d73 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:28.830296 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1104/2243] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:34:28 2022] PUT /identity/v3/projects/1d4148b35eac43a0bca89c631559ddfe/users/f1480d45394b4ae294d0a4b1b59af0d1/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:34:28.844184 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a72b77-4440-49b5-a7aa-2a57e343b7f3 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:28.844813 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a72b77-4440-49b5-a7aa-2a57e343b7f3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:28.845196 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a72b77-4440-49b5-a7aa-2a57e343b7f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:28.884547 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-49a72b77-4440-49b5-a7aa-2a57e343b7f3 None None] MFA Rules not processed for user `f1480d45394b4ae294d0a4b1b59af0d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:34:28.886886 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-49a72b77-4440-49b5-a7aa-2a57e343b7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:28.995141 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1140/2244] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:34:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:34:29.010991 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada359c7-306d-43cf-9631-c53c1da90d73 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:29.013015 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ada359c7-306d-43cf-9631-c53c1da90d73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:29.059897 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:34:29.163694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada359c7-306d-43cf-9631-c53c1da90d73 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:29.167679 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435acd49-f95f-443d-8b24-32656fd9e261 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:29.168343 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435acd49-f95f-443d-8b24-32656fd9e261 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:29.168633 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435acd49-f95f-443d-8b24-32656fd9e261 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:29.168911 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435acd49-f95f-443d-8b24-32656fd9e261 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:29.179287 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435acd49-f95f-443d-8b24-32656fd9e261 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:29.180783 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435acd49-f95f-443d-8b24-32656fd9e261 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:29.191642 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1105/2245] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:34:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:34:29.226306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a72b77-4440-49b5-a7aa-2a57e343b7f3 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:29.248193 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:34:29.317257 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a72b77-4440-49b5-a7aa-2a57e343b7f3 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:29.328041 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b7146-7678-480b-ae63-92cd1fba1fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:29.332961 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b7146-7678-480b-ae63-92cd1fba1fb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:29.332961 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b7146-7678-480b-ae63-92cd1fba1fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:29.332961 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b7146-7678-480b-ae63-92cd1fba1fb6 admin admin] PATH_INFO: `/v3/users/f1480d45394b4ae294d0a4b1b59af0d1` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:29.333848 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b7146-7678-480b-ae63-92cd1fba1fb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1480d45394b4ae294d0a4b1b59af0d1)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:29.337746 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b7146-7678-480b-ae63-92cd1fba1fb6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:29.402947 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4d5b7146-7678-480b-ae63-92cd1fba1fb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1480d45394b4ae294d0a4b1b59af0d1'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:29.429184 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4d5b7146-7678-480b-ae63-92cd1fba1fb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1480d45394b4ae294d0a4b1b59af0d1'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:29.429927 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4d5b7146-7678-480b-ae63-92cd1fba1fb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1480d45394b4ae294d0a4b1b59af0d1'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:29.430571 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4d5b7146-7678-480b-ae63-92cd1fba1fb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1480d45394b4ae294d0a4b1b59af0d1'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:29.447196 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4d5b7146-7678-480b-ae63-92cd1fba1fb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1480d45394b4ae294d0a4b1b59af0d1'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:29.470681 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1141/2246] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:34:29 2022] DELETE /identity/v3/users/f1480d45394b4ae294d0a4b1b59af0d1 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:34:29.478519 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618eb279-5eb1-4fe5-9e8e-1375dd858598 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:29.478519 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618eb279-5eb1-4fe5-9e8e-1375dd858598 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:29.478519 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618eb279-5eb1-4fe5-9e8e-1375dd858598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:29.514300 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-618eb279-5eb1-4fe5-9e8e-1375dd858598 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:34:29.517967 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-618eb279-5eb1-4fe5-9e8e-1375dd858598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:29.654501 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1106/2247] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:34:29.666739 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b7146-7678-480b-ae63-92cd1fba1fb6 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:29.671282 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4d5b7146-7678-480b-ae63-92cd1fba1fb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:29.746160 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b7146-7678-480b-ae63-92cd1fba1fb6 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:29.750406 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eece68-42d5-4f60-b82e-47d833033b2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:29.751095 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eece68-42d5-4f60-b82e-47d833033b2b service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:29.751375 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eece68-42d5-4f60-b82e-47d833033b2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:29.751825 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eece68-42d5-4f60-b82e-47d833033b2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:29.769300 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eece68-42d5-4f60-b82e-47d833033b2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:29.770967 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eece68-42d5-4f60-b82e-47d833033b2b service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:29.783541 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1142/2248] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:34:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:34:30.665331 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618eb279-5eb1-4fe5-9e8e-1375dd858598 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:30.668399 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-618eb279-5eb1-4fe5-9e8e-1375dd858598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:30.694801 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618eb279-5eb1-4fe5-9e8e-1375dd858598 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:30.706042 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e814163d-4e36-4411-a9fd-2de6ef99a8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:30.707551 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814163d-4e36-4411-a9fd-2de6ef99a8e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:30.708039 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814163d-4e36-4411-a9fd-2de6ef99a8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:30.708520 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814163d-4e36-4411-a9fd-2de6ef99a8e8 admin admin] PATH_INFO: `/v3/projects/1d4148b35eac43a0bca89c631559ddfe` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:30.710845 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e814163d-4e36-4411-a9fd-2de6ef99a8e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d4148b35eac43a0bca89c631559ddfe)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:30.714766 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e814163d-4e36-4411-a9fd-2de6ef99a8e8 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:30.774447 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e814163d-4e36-4411-a9fd-2de6ef99a8e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d4148b35eac43a0bca89c631559ddfe', 'request_id': 'req-e814163d-4e36-4411-a9fd-2de6ef99a8e8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:30.781778 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e814163d-4e36-4411-a9fd-2de6ef99a8e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d4148b35eac43a0bca89c631559ddfe', 'request_id': 'req-e814163d-4e36-4411-a9fd-2de6ef99a8e8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:30.790829 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e814163d-4e36-4411-a9fd-2de6ef99a8e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d4148b35eac43a0bca89c631559ddfe', 'request_id': 'req-e814163d-4e36-4411-a9fd-2de6ef99a8e8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:30.805951 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e814163d-4e36-4411-a9fd-2de6ef99a8e8 admin admin] The token cache is being invalidate because project 1d4148b35eac43a0bca89c631559ddfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:34:30.806446 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e814163d-4e36-4411-a9fd-2de6ef99a8e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:30.807568 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e814163d-4e36-4411-a9fd-2de6ef99a8e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:30.810311 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1107/2249] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:34:30 2022] DELETE /identity/v3/projects/1d4148b35eac43a0bca89c631559ddfe => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:34:30.817783 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9237c-75b7-4cef-91bf-2e5044c7682c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:30.818428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9237c-75b7-4cef-91bf-2e5044c7682c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:30.818533 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9237c-75b7-4cef-91bf-2e5044c7682c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:30.838044 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-bbd9237c-75b7-4cef-91bf-2e5044c7682c None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:34:30.839438 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-bbd9237c-75b7-4cef-91bf-2e5044c7682c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:30.906956 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:34:30.952044 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1143/2250] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:34:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:34:30.960356 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16bf3a4-79ba-497c-8798-508e17e4e0b4 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:30.960704 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16bf3a4-79ba-497c-8798-508e17e4e0b4 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:30.961013 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16bf3a4-79ba-497c-8798-508e17e4e0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:30.986914 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-c16bf3a4-79ba-497c-8798-508e17e4e0b4 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:34:30.988448 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c16bf3a4-79ba-497c-8798-508e17e4e0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:31.003050 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1108/2251] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:34:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:34:31.009097 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd9237c-75b7-4cef-91bf-2e5044c7682c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:31.023504 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd9237c-75b7-4cef-91bf-2e5044c7682c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:31.032253 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fbc829-7923-4747-ad78-ea225e5dccf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:31.033603 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fbc829-7923-4747-ad78-ea225e5dccf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:31.034185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fbc829-7923-4747-ad78-ea225e5dccf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:31.034790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fbc829-7923-4747-ad78-ea225e5dccf8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:31.036347 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fbc829-7923-4747-ad78-ea225e5dccf8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:34:31.036976 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fbc829-7923-4747-ad78-ea225e5dccf8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:31.042590 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fbc829-7923-4747-ad78-ea225e5dccf8 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:31.075285 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16bf3a4-79ba-497c-8798-508e17e4e0b4 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:31.080350 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1144/2252] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:34:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:34:31.086106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16bf3a4-79ba-497c-8798-508e17e4e0b4 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:31.092316 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564780cc-c892-4e0b-a17d-eb72b0efefd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:31.093165 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564780cc-c892-4e0b-a17d-eb72b0efefd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:31.093453 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564780cc-c892-4e0b-a17d-eb72b0efefd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:31.093892 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564780cc-c892-4e0b-a17d-eb72b0efefd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:31.094661 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564780cc-c892-4e0b-a17d-eb72b0efefd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:31.096656 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564780cc-c892-4e0b-a17d-eb72b0efefd6 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:31.138686 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:34:31.161894 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1109/2253] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:34:31 2022] POST /identity/v3/projects => generated 385 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:34:31.162592 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fbc829-7923-4747-ad78-ea225e5dccf8 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:31.177416 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fbc829-7923-4747-ad78-ea225e5dccf8 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:31.273176 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd7d5cf-8784-4c70-ba98-e29affd3dc16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:31.273883 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7d5cf-8784-4c70-ba98-e29affd3dc16 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:31.274169 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7d5cf-8784-4c70-ba98-e29affd3dc16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:31.274486 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7d5cf-8784-4c70-ba98-e29affd3dc16 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:31.275920 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd7d5cf-8784-4c70-ba98-e29affd3dc16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:31.277692 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd7d5cf-8784-4c70-ba98-e29affd3dc16 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:31.318181 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1145/2254] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:34:31 2022] POST /identity/v3/users => generated 293 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:34:31.322369 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564780cc-c892-4e0b-a17d-eb72b0efefd6 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:31.342031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564780cc-c892-4e0b-a17d-eb72b0efefd6 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:31.347763 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e33ea-f8fd-4dc9-b3a5-0b508ef423a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:31.348464 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e33ea-f8fd-4dc9-b3a5-0b508ef423a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:31.348749 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e33ea-f8fd-4dc9-b3a5-0b508ef423a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:31.349028 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e33ea-f8fd-4dc9-b3a5-0b508ef423a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:31.349771 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8e33ea-f8fd-4dc9-b3a5-0b508ef423a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:31.351577 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8e33ea-f8fd-4dc9-b3a5-0b508ef423a4 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:31.364541 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1110/2255] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 21:34:31.374058 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd7d5cf-8784-4c70-ba98-e29affd3dc16 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:31.394641 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd7d5cf-8784-4c70-ba98-e29affd3dc16 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:31.402594 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e39a0c-decc-4158-a59a-ff4da672ce2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:31.403450 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e39a0c-decc-4158-a59a-ff4da672ce2c admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:31.403909 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e39a0c-decc-4158-a59a-ff4da672ce2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:31.404327 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e39a0c-decc-4158-a59a-ff4da672ce2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:31.405392 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e39a0c-decc-4158-a59a-ff4da672ce2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:31.407674 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e39a0c-decc-4158-a59a-ff4da672ce2c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:31.430997 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1146/2256] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:34:31.434930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e33ea-f8fd-4dc9-b3a5-0b508ef423a4 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:31.450643 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e33ea-f8fd-4dc9-b3a5-0b508ef423a4 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:31.451140 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e39a0c-decc-4158-a59a-ff4da672ce2c admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:31.455786 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-96e39a0c-decc-4158-a59a-ff4da672ce2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:31.458653 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2040689d-465c-4cc2-aca9-1501d29e316f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:31.460171 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040689d-465c-4cc2-aca9-1501d29e316f admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:31.460645 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040689d-465c-4cc2-aca9-1501d29e316f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:31.461120 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040689d-465c-4cc2-aca9-1501d29e316f admin admin] PATH_INFO: `/v3/projects/3fcf03712c6f4926a2a95dfd0519536a/users/1664dba0cd9b4d9da1e4b78691b8567e/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:31.478470 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e39a0c-decc-4158-a59a-ff4da672ce2c admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:31.483651 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2040689d-465c-4cc2-aca9-1501d29e316f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fcf03712c6f4926a2a95dfd0519536a, user_id=1664dba0cd9b4d9da1e4b78691b8567e, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:31.486631 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2040689d-465c-4cc2-aca9-1501d29e316f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:31.487371 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de962b-4f6a-4871-8f1f-0a8b507eb5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:31.488422 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de962b-4f6a-4871-8f1f-0a8b507eb5ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:31.488702 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de962b-4f6a-4871-8f1f-0a8b507eb5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:31.489085 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de962b-4f6a-4871-8f1f-0a8b507eb5ea admin admin] PATH_INFO: `/v3/users/c5dc7ff3a5624d8ca2cf116f8f9f0b2e` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:31.491113 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1de962b-4f6a-4871-8f1f-0a8b507eb5ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5dc7ff3a5624d8ca2cf116f8f9f0b2e)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:31.493404 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1de962b-4f6a-4871-8f1f-0a8b507eb5ea admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:31.504610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1111/2257] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:34:31 2022] PUT /identity/v3/projects/3fcf03712c6f4926a2a95dfd0519536a/users/1664dba0cd9b4d9da1e4b78691b8567e/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:34:31.509810 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e93140-1ada-4b62-ba7c-021278c3103a None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:31.510095 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e93140-1ada-4b62-ba7c-021278c3103a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:31.510407 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e93140-1ada-4b62-ba7c-021278c3103a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:31.532506 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-22e93140-1ada-4b62-ba7c-021278c3103a None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:34:31.534004 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-22e93140-1ada-4b62-ba7c-021278c3103a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:31.567165 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e1de962b-4f6a-4871-8f1f-0a8b507eb5ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5dc7ff3a5624d8ca2cf116f8f9f0b2e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:31.581801 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e1de962b-4f6a-4871-8f1f-0a8b507eb5ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5dc7ff3a5624d8ca2cf116f8f9f0b2e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:31.582804 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e1de962b-4f6a-4871-8f1f-0a8b507eb5ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5dc7ff3a5624d8ca2cf116f8f9f0b2e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:31.583649 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e1de962b-4f6a-4871-8f1f-0a8b507eb5ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5dc7ff3a5624d8ca2cf116f8f9f0b2e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:31.594574 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e1de962b-4f6a-4871-8f1f-0a8b507eb5ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5dc7ff3a5624d8ca2cf116f8f9f0b2e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:31.620623 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1147/2258] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:34:31 2022] DELETE /identity/v3/users/c5dc7ff3a5624d8ca2cf116f8f9f0b2e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:34:31.641414 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1112/2259] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:34:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:34:31.657880 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de962b-4f6a-4871-8f1f-0a8b507eb5ea admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:31.660342 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e1de962b-4f6a-4871-8f1f-0a8b507eb5ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:31.713792 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:34:31.785419 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de962b-4f6a-4871-8f1f-0a8b507eb5ea admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:31.790337 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d4bbd9-3815-4ded-88be-b97f07802146 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:31.791272 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4bbd9-3815-4ded-88be-b97f07802146 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:31.791804 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4bbd9-3815-4ded-88be-b97f07802146 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:31.792385 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4bbd9-3815-4ded-88be-b97f07802146 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:31.925359 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d4bbd9-3815-4ded-88be-b97f07802146 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:31.927571 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d4bbd9-3815-4ded-88be-b97f07802146 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:31.948568 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1148/2260] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:34:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:34:31.980109 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e93140-1ada-4b62-ba7c-021278c3103a None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:31.983042 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-22e93140-1ada-4b62-ba7c-021278c3103a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:32.019501 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e93140-1ada-4b62-ba7c-021278c3103a None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:32.031325 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f83f33d-9e57-4230-9447-171e1c5891a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:32.033715 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83f33d-9e57-4230-9447-171e1c5891a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:32.034291 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83f33d-9e57-4230-9447-171e1c5891a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:32.034802 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83f33d-9e57-4230-9447-171e1c5891a6 admin admin] PATH_INFO: `/v3/projects/2fb03711de75418e89907ca315beaadc` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:32.037625 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f83f33d-9e57-4230-9447-171e1c5891a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fb03711de75418e89907ca315beaadc)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:32.041860 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f83f33d-9e57-4230-9447-171e1c5891a6 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:32.128309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8f83f33d-9e57-4230-9447-171e1c5891a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fb03711de75418e89907ca315beaadc', 'request_id': 'req-8f83f33d-9e57-4230-9447-171e1c5891a6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:32.149248 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8f83f33d-9e57-4230-9447-171e1c5891a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fb03711de75418e89907ca315beaadc', 'request_id': 'req-8f83f33d-9e57-4230-9447-171e1c5891a6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:32.163294 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8f83f33d-9e57-4230-9447-171e1c5891a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fb03711de75418e89907ca315beaadc', 'request_id': 'req-8f83f33d-9e57-4230-9447-171e1c5891a6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:32.190672 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8f83f33d-9e57-4230-9447-171e1c5891a6 admin admin] The token cache is being invalidate because project 2fb03711de75418e89907ca315beaadc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:34:32.191245 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8f83f33d-9e57-4230-9447-171e1c5891a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:32.192559 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8f83f33d-9e57-4230-9447-171e1c5891a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:32.196043 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1113/2261] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:34:31 2022] DELETE /identity/v3/projects/2fb03711de75418e89907ca315beaadc => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:34:32.200752 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d4bbd9-3815-4ded-88be-b97f07802146 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:32.220941 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d4bbd9-3815-4ded-88be-b97f07802146 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:32.322667 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71526d40-c000-4582-9386-89ede24a86db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:32.324599 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71526d40-c000-4582-9386-89ede24a86db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:32.325178 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71526d40-c000-4582-9386-89ede24a86db admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:32.325667 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71526d40-c000-4582-9386-89ede24a86db admin admin] PATH_INFO: `/v3/users/d9820c3bc153466cadfcbaa2bb9233c9` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:32.328718 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71526d40-c000-4582-9386-89ede24a86db admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9820c3bc153466cadfcbaa2bb9233c9)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:32.331785 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71526d40-c000-4582-9386-89ede24a86db admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:32.434435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-71526d40-c000-4582-9386-89ede24a86db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9820c3bc153466cadfcbaa2bb9233c9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:32.454460 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-71526d40-c000-4582-9386-89ede24a86db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9820c3bc153466cadfcbaa2bb9233c9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:32.455309 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-71526d40-c000-4582-9386-89ede24a86db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9820c3bc153466cadfcbaa2bb9233c9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:32.455923 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-71526d40-c000-4582-9386-89ede24a86db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9820c3bc153466cadfcbaa2bb9233c9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:32.464508 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:34:32.472468 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-71526d40-c000-4582-9386-89ede24a86db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9820c3bc153466cadfcbaa2bb9233c9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:32.494560 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1149/2262] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:34:32 2022] DELETE /identity/v3/users/d9820c3bc153466cadfcbaa2bb9233c9 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:34:32.753604 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f83f33d-9e57-4230-9447-171e1c5891a6 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:32.755686 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8f83f33d-9e57-4230-9447-171e1c5891a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:32.779042 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f83f33d-9e57-4230-9447-171e1c5891a6 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:32.878341 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:34:32.909150 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b2e3e4-0384-4af6-acdd-9b736830b4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:32.910695 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2e3e4-0384-4af6-acdd-9b736830b4ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:32.911003 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2e3e4-0384-4af6-acdd-9b736830b4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:32.911290 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2e3e4-0384-4af6-acdd-9b736830b4ad admin admin] PATH_INFO: `/v3/projects/11f59c8c907e42049a28d8c5bb8eba97` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:32.912643 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b2e3e4-0384-4af6-acdd-9b736830b4ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=11f59c8c907e42049a28d8c5bb8eba97)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:32.914783 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b2e3e4-0384-4af6-acdd-9b736830b4ad admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:32.988845 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-16b2e3e4-0384-4af6-acdd-9b736830b4ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11f59c8c907e42049a28d8c5bb8eba97', 'request_id': 'req-16b2e3e4-0384-4af6-acdd-9b736830b4ad'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:32.996502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-16b2e3e4-0384-4af6-acdd-9b736830b4ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11f59c8c907e42049a28d8c5bb8eba97', 'request_id': 'req-16b2e3e4-0384-4af6-acdd-9b736830b4ad'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:33.010535 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-16b2e3e4-0384-4af6-acdd-9b736830b4ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11f59c8c907e42049a28d8c5bb8eba97', 'request_id': 'req-16b2e3e4-0384-4af6-acdd-9b736830b4ad'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:33.029699 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-16b2e3e4-0384-4af6-acdd-9b736830b4ad admin admin] The token cache is being invalidate because project 11f59c8c907e42049a28d8c5bb8eba97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:34:33.030199 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-16b2e3e4-0384-4af6-acdd-9b736830b4ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:33.031139 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-16b2e3e4-0384-4af6-acdd-9b736830b4ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:33.033328 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1114/2263] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:34:32 2022] DELETE /identity/v3/projects/11f59c8c907e42049a28d8c5bb8eba97 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:34:33.041073 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb106ae-4515-41d1-aa34-69982690f60a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:33.041073 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb106ae-4515-41d1-aa34-69982690f60a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:33.041073 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb106ae-4515-41d1-aa34-69982690f60a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:33.065874 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-beb106ae-4515-41d1-aa34-69982690f60a None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:34:33.067540 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-beb106ae-4515-41d1-aa34-69982690f60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:33.149747 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1150/2264] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:34:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:34:33.159537 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3149726-7bd5-4476-aa37-b1ab6547d97c None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:33.159926 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3149726-7bd5-4476-aa37-b1ab6547d97c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:33.160217 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3149726-7bd5-4476-aa37-b1ab6547d97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:33.190922 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-a3149726-7bd5-4476-aa37-b1ab6547d97c None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:34:33.192468 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a3149726-7bd5-4476-aa37-b1ab6547d97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:33.206042 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1115/2265] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:34:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:34:33.207526 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb106ae-4515-41d1-aa34-69982690f60a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:33.222670 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb106ae-4515-41d1-aa34-69982690f60a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:33.232713 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0766fc11-98c8-4189-abf8-9fd60ba9e448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:33.233980 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766fc11-98c8-4189-abf8-9fd60ba9e448 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:33.234752 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766fc11-98c8-4189-abf8-9fd60ba9e448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:33.235383 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766fc11-98c8-4189-abf8-9fd60ba9e448 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:33.236416 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0766fc11-98c8-4189-abf8-9fd60ba9e448 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:34:33.236994 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0766fc11-98c8-4189-abf8-9fd60ba9e448 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:33.241206 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0766fc11-98c8-4189-abf8-9fd60ba9e448 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:33.262588 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1151/2266] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:34:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:34:33.267085 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3149726-7bd5-4476-aa37-b1ab6547d97c None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:33.278915 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3149726-7bd5-4476-aa37-b1ab6547d97c None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:33.285080 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb08ec-ad23-4b58-92ec-0760c2bbc276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:33.285766 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb08ec-ad23-4b58-92ec-0760c2bbc276 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:33.286102 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb08ec-ad23-4b58-92ec-0760c2bbc276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:33.286423 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb08ec-ad23-4b58-92ec-0760c2bbc276 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:33.287236 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb08ec-ad23-4b58-92ec-0760c2bbc276 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:33.289364 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb08ec-ad23-4b58-92ec-0760c2bbc276 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:33.337813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1116/2267] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:34:33 2022] POST /identity/v3/projects => generated 385 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:34:33.342808 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0766fc11-98c8-4189-abf8-9fd60ba9e448 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:33.357240 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0766fc11-98c8-4189-abf8-9fd60ba9e448 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:33.441167 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc2029c-a642-47b7-a167-d439bb77bc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:33.441783 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2029c-a642-47b7-a167-d439bb77bc19 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:33.442069 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2029c-a642-47b7-a167-d439bb77bc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:33.442390 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2029c-a642-47b7-a167-d439bb77bc19 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:33.443116 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc2029c-a642-47b7-a167-d439bb77bc19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:33.445124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc2029c-a642-47b7-a167-d439bb77bc19 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:33.469449 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1152/2268] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:34:33 2022] POST /identity/v3/users => generated 293 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:34:33.475150 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb08ec-ad23-4b58-92ec-0760c2bbc276 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:33.494341 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb08ec-ad23-4b58-92ec-0760c2bbc276 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:33.503019 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f9d4b-97f8-4861-9312-52e309de1f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:33.504166 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f9d4b-97f8-4861-9312-52e309de1f43 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:33.504654 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f9d4b-97f8-4861-9312-52e309de1f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:33.505154 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f9d4b-97f8-4861-9312-52e309de1f43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:33.506558 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f9d4b-97f8-4861-9312-52e309de1f43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:33.510290 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f9d4b-97f8-4861-9312-52e309de1f43 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:33.534379 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1117/2269] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:34:33.538148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc2029c-a642-47b7-a167-d439bb77bc19 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:33.555556 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc2029c-a642-47b7-a167-d439bb77bc19 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:33.568427 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c250182-a929-41b8-b159-0138ad593133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:33.569637 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c250182-a929-41b8-b159-0138ad593133 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:33.570325 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c250182-a929-41b8-b159-0138ad593133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:33.570702 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c250182-a929-41b8-b159-0138ad593133 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:33.572008 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c250182-a929-41b8-b159-0138ad593133 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:33.574836 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c250182-a929-41b8-b159-0138ad593133 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:33.595102 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1153/2270] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:34:33.601505 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f9d4b-97f8-4861-9312-52e309de1f43 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:33.619632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f9d4b-97f8-4861-9312-52e309de1f43 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:33.635924 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39642c4e-cb34-49e9-8f62-87808fc5a3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:33.637529 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39642c4e-cb34-49e9-8f62-87808fc5a3b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:33.637920 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39642c4e-cb34-49e9-8f62-87808fc5a3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:33.638335 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39642c4e-cb34-49e9-8f62-87808fc5a3b8 admin admin] PATH_INFO: `/v3/projects/60ba2171e8a2449bbf59e191b261ac57/users/3346f90be94247a2bdd8b354e631bd16/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:33.684501 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39642c4e-cb34-49e9-8f62-87808fc5a3b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60ba2171e8a2449bbf59e191b261ac57, user_id=3346f90be94247a2bdd8b354e631bd16, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:33.689294 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39642c4e-cb34-49e9-8f62-87808fc5a3b8 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:33.713726 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1118/2271] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:34:33 2022] PUT /identity/v3/projects/60ba2171e8a2449bbf59e191b261ac57/users/3346f90be94247a2bdd8b354e631bd16/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:34:33.724499 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee48acd-e0c8-4287-8af6-8ea393dc9213 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:33.724898 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee48acd-e0c8-4287-8af6-8ea393dc9213 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:33.725221 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee48acd-e0c8-4287-8af6-8ea393dc9213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:33.764614 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-5ee48acd-e0c8-4287-8af6-8ea393dc9213 None None] MFA Rules not processed for user `3346f90be94247a2bdd8b354e631bd16`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:34:33.766391 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5ee48acd-e0c8-4287-8af6-8ea393dc9213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:33.831430 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:34:33.873649 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1154/2272] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:34:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:34:33.880224 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39642c4e-cb34-49e9-8f62-87808fc5a3b8 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:34.009494 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39642c4e-cb34-49e9-8f62-87808fc5a3b8 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:34.014739 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f4b71-93f1-47b6-a2f8-b671abb3815e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:34.015657 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f4b71-93f1-47b6-a2f8-b671abb3815e admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:34.015940 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f4b71-93f1-47b6-a2f8-b671abb3815e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:34.016217 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f4b71-93f1-47b6-a2f8-b671abb3815e admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:34.016995 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1f4b71-93f1-47b6-a2f8-b671abb3815e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:34.020122 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1f4b71-93f1-47b6-a2f8-b671abb3815e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:34.046563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:34:34.057895 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1119/2273] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:34:33 2022] POST /identity/v3/projects => generated 383 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:34:34.064358 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee48acd-e0c8-4287-8af6-8ea393dc9213 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:34.081757 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee48acd-e0c8-4287-8af6-8ea393dc9213 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:34.225863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa09ce-6f0d-449a-9c89-39538c526022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:34.226655 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa09ce-6f0d-449a-9c89-39538c526022 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:34.227007 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa09ce-6f0d-449a-9c89-39538c526022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:34.227360 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa09ce-6f0d-449a-9c89-39538c526022 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:34.228170 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aa09ce-6f0d-449a-9c89-39538c526022 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:34.230250 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aa09ce-6f0d-449a-9c89-39538c526022 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:34.270162 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1155/2274] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:34:34 2022] POST /identity/v3/users => generated 292 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:34:34.274532 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f4b71-93f1-47b6-a2f8-b671abb3815e admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:34.289943 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f4b71-93f1-47b6-a2f8-b671abb3815e admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:34.294522 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a8d727-a276-4ee2-b90c-9782aa17d75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:34.295236 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8d727-a276-4ee2-b90c-9782aa17d75a admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:34.295517 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8d727-a276-4ee2-b90c-9782aa17d75a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:34.297363 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8d727-a276-4ee2-b90c-9782aa17d75a admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:34.298273 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a8d727-a276-4ee2-b90c-9782aa17d75a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:34.300021 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a8d727-a276-4ee2-b90c-9782aa17d75a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:34.317987 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1120/2275] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:34:34.322338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa09ce-6f0d-449a-9c89-39538c526022 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:34.351831 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa09ce-6f0d-449a-9c89-39538c526022 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:34.363630 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97687681-8549-46c4-b00a-c18954be7b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:34.365419 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97687681-8549-46c4-b00a-c18954be7b2c admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:34.366085 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97687681-8549-46c4-b00a-c18954be7b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:34.367176 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97687681-8549-46c4-b00a-c18954be7b2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:34.369172 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97687681-8549-46c4-b00a-c18954be7b2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:34.372653 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97687681-8549-46c4-b00a-c18954be7b2c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:34.410016 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a8d727-a276-4ee2-b90c-9782aa17d75a admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:34.410700 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1156/2276] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:34:34.425197 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a8d727-a276-4ee2-b90c-9782aa17d75a admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:34.434587 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e64200-e9e7-483f-87c0-0d3505a2d8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:34.436275 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e64200-e9e7-483f-87c0-0d3505a2d8a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:34.436918 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e64200-e9e7-483f-87c0-0d3505a2d8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:34.437527 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e64200-e9e7-483f-87c0-0d3505a2d8a4 admin admin] PATH_INFO: `/v3/projects/5bb4fd090a8c40aab3a565a6d1596910/users/8a665e53d92a4740961253eacda24b55/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:34.474035 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e64200-e9e7-483f-87c0-0d3505a2d8a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bb4fd090a8c40aab3a565a6d1596910, user_id=8a665e53d92a4740961253eacda24b55, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:34.478461 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e64200-e9e7-483f-87c0-0d3505a2d8a4 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:34.498472 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1121/2277] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:34:34 2022] PUT /identity/v3/projects/5bb4fd090a8c40aab3a565a6d1596910/users/8a665e53d92a4740961253eacda24b55/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:34:34.519208 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f59b5-162f-4586-8920-7d2820b426eb None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:34.519598 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f59b5-162f-4586-8920-7d2820b426eb None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:34.519913 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f59b5-162f-4586-8920-7d2820b426eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:34.562642 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d24f59b5-162f-4586-8920-7d2820b426eb None None] MFA Rules not processed for user `8a665e53d92a4740961253eacda24b55`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:34:34.564822 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d24f59b5-162f-4586-8920-7d2820b426eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:34.668740 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1157/2278] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:34:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:34:34.673559 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e64200-e9e7-483f-87c0-0d3505a2d8a4 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:34.760089 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e64200-e9e7-483f-87c0-0d3505a2d8a4 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:34.766800 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffad750-4988-4e31-a959-c5b3a0627eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:34.767008 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffad750-4988-4e31-a959-c5b3a0627eef admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:34.767251 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffad750-4988-4e31-a959-c5b3a0627eef admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:34.767527 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffad750-4988-4e31-a959-c5b3a0627eef admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:34.768252 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffad750-4988-4e31-a959-c5b3a0627eef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:34.770219 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffad750-4988-4e31-a959-c5b3a0627eef admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:34.802112 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1122/2279] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:34:34 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:34:34.806454 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f59b5-162f-4586-8920-7d2820b426eb None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:34.819978 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f59b5-162f-4586-8920-7d2820b426eb None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:34.905834 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:34:34.923725 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19bd1b-50b5-470a-b54f-a44e8d15e070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:34.924459 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19bd1b-50b5-470a-b54f-a44e8d15e070 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:34.924752 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19bd1b-50b5-470a-b54f-a44e8d15e070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:34.925050 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19bd1b-50b5-470a-b54f-a44e8d15e070 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:34.926457 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d19bd1b-50b5-470a-b54f-a44e8d15e070 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:34.928462 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d19bd1b-50b5-470a-b54f-a44e8d15e070 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:34.955372 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1158/2280] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:34:34 2022] POST /identity/v3/users => generated 299 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:34:34.959240 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffad750-4988-4e31-a959-c5b3a0627eef admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:34.978014 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffad750-4988-4e31-a959-c5b3a0627eef admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:34.983668 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668fba4-f403-420c-9342-dde2e7e482c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:34.984772 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668fba4-f403-420c-9342-dde2e7e482c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:34.985066 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668fba4-f403-420c-9342-dde2e7e482c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:34.985350 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668fba4-f403-420c-9342-dde2e7e482c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:34.986333 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a668fba4-f403-420c-9342-dde2e7e482c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:34.988179 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a668fba4-f403-420c-9342-dde2e7e482c2 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:35.004827 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1123/2281] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 21:34:35.009195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19bd1b-50b5-470a-b54f-a44e8d15e070 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:35.023676 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19bd1b-50b5-470a-b54f-a44e8d15e070 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:35.036367 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5631c6c-65e8-4688-98fc-9011c654f2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:35.037052 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5631c6c-65e8-4688-98fc-9011c654f2be admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:35.037338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5631c6c-65e8-4688-98fc-9011c654f2be admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:35.037710 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5631c6c-65e8-4688-98fc-9011c654f2be admin admin] PATH_INFO: `/v3/projects/009fdb22845142b690966249f44116f2/users/f434028bee6b4d328cbf1ce06a857c55/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:35.058315 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5631c6c-65e8-4688-98fc-9011c654f2be admin admin] RBAC: Authorizing `identity:create_grant(project_id=009fdb22845142b690966249f44116f2, user_id=f434028bee6b4d328cbf1ce06a857c55, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:35.061237 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5631c6c-65e8-4688-98fc-9011c654f2be admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:35.081383 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1159/2282] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:34:35 2022] PUT /identity/v3/projects/009fdb22845142b690966249f44116f2/users/f434028bee6b4d328cbf1ce06a857c55/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:34:35.085378 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668fba4-f403-420c-9342-dde2e7e482c2 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:35.104444 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668fba4-f403-420c-9342-dde2e7e482c2 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:35.186833 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a842c9-3b6a-490b-b137-e81164d8875b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:35.187594 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a842c9-3b6a-490b-b137-e81164d8875b admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:35.187885 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a842c9-3b6a-490b-b137-e81164d8875b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:35.188166 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a842c9-3b6a-490b-b137-e81164d8875b admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:35.188965 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a842c9-3b6a-490b-b137-e81164d8875b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:35.190568 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a842c9-3b6a-490b-b137-e81164d8875b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:35.202857 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1124/2283] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:34:35.211395 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5631c6c-65e8-4688-98fc-9011c654f2be admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:35.230485 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5631c6c-65e8-4688-98fc-9011c654f2be admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:35.246645 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f23db0-235b-4c8d-aeca-12fe4cffb7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:35.247702 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f23db0-235b-4c8d-aeca-12fe4cffb7c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:35.248065 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f23db0-235b-4c8d-aeca-12fe4cffb7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:35.248428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f23db0-235b-4c8d-aeca-12fe4cffb7c8 admin admin] PATH_INFO: `/v3/projects/009fdb22845142b690966249f44116f2/users/f434028bee6b4d328cbf1ce06a857c55/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:35.251387 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f23db0-235b-4c8d-aeca-12fe4cffb7c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=009fdb22845142b690966249f44116f2, user_id=f434028bee6b4d328cbf1ce06a857c55, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:35.256088 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f23db0-235b-4c8d-aeca-12fe4cffb7c8 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:35.283466 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1160/2284] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:34:35 2022] PUT /identity/v3/projects/009fdb22845142b690966249f44116f2/users/f434028bee6b4d328cbf1ce06a857c55/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:34:35.297424 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba4511-1b57-44b9-9a5c-43bdce7374aa None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:35.298636 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba4511-1b57-44b9-9a5c-43bdce7374aa None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:35.299033 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba4511-1b57-44b9-9a5c-43bdce7374aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:35.337170 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-99ba4511-1b57-44b9-9a5c-43bdce7374aa None None] MFA Rules not processed for user `f434028bee6b4d328cbf1ce06a857c55`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:34:35.338630 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-99ba4511-1b57-44b9-9a5c-43bdce7374aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:35.369786 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:34:35.498777 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1125/2285] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:34:35 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:34:35.511607 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f23db0-235b-4c8d-aeca-12fe4cffb7c8 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:35.514352 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e4f23db0-235b-4c8d-aeca-12fe4cffb7c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:35.667697 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f23db0-235b-4c8d-aeca-12fe4cffb7c8 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:35.687479 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f5229-cbd9-4b23-819c-b8d75e5f0e05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:35.687780 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f5229-cbd9-4b23-819c-b8d75e5f0e05 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:35.688092 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f5229-cbd9-4b23-819c-b8d75e5f0e05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:35.688282 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f5229-cbd9-4b23-819c-b8d75e5f0e05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:35.783759 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f5229-cbd9-4b23-819c-b8d75e5f0e05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:35.786203 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f5229-cbd9-4b23-819c-b8d75e5f0e05 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:35.802489 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1161/2286] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:34:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:34:38.030558 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94227f28-0d11-4724-9434-c1927be0b4d3 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:38.031153 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94227f28-0d11-4724-9434-c1927be0b4d3 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:38.031153 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94227f28-0d11-4724-9434-c1927be0b4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:38.139435 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-94227f28-0d11-4724-9434-c1927be0b4d3 None None] MFA Rules not processed for user `1664dba0cd9b4d9da1e4b78691b8567e`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:34:38.147355 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-94227f28-0d11-4724-9434-c1927be0b4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:38.291799 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1126/2287] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:34:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:34:38.297969 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f5229-cbd9-4b23-819c-b8d75e5f0e05 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:38.304271 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-de7f5229-cbd9-4b23-819c-b8d75e5f0e05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:38.463369 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:34:38.470072 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f5229-cbd9-4b23-819c-b8d75e5f0e05 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:38.479487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30291037-d1e2-4772-ba8e-e03e141047f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:38.480882 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30291037-d1e2-4772-ba8e-e03e141047f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:38.481864 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30291037-d1e2-4772-ba8e-e03e141047f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:38.482624 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30291037-d1e2-4772-ba8e-e03e141047f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:38.483736 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30291037-d1e2-4772-ba8e-e03e141047f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:38.485957 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30291037-d1e2-4772-ba8e-e03e141047f1 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:38.558341 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1162/2288] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:34:38 2022] POST /identity/v3/projects => generated 385 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:34:38.563486 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94227f28-0d11-4724-9434-c1927be0b4d3 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:38.603723 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94227f28-0d11-4724-9434-c1927be0b4d3 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:38.747583 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f0f834-1acc-4ef4-b022-3f7b60913aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:38.748743 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f0f834-1acc-4ef4-b022-3f7b60913aa9 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:38.749262 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f0f834-1acc-4ef4-b022-3f7b60913aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:38.750077 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f0f834-1acc-4ef4-b022-3f7b60913aa9 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:38.751622 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f0f834-1acc-4ef4-b022-3f7b60913aa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:38.758107 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f0f834-1acc-4ef4-b022-3f7b60913aa9 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:38.782170 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30291037-d1e2-4772-ba8e-e03e141047f1 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:38.802867 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1127/2289] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:34:38 2022] POST /identity/v3/users => generated 299 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:34:38.822375 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f0f834-1acc-4ef4-b022-3f7b60913aa9 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:38.840845 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f0f834-1acc-4ef4-b022-3f7b60913aa9 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:38.860086 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2a892-4496-4e7b-80fd-040abde4b2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:38.861074 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2a892-4496-4e7b-80fd-040abde4b2ba admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:38.862775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2a892-4496-4e7b-80fd-040abde4b2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:38.863251 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2a892-4496-4e7b-80fd-040abde4b2ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:38.864238 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2a892-4496-4e7b-80fd-040abde4b2ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:38.866026 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2a892-4496-4e7b-80fd-040abde4b2ba admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:38.880175 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1128/2290] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 21:34:38.886717 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2a892-4496-4e7b-80fd-040abde4b2ba admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:38.903404 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2a892-4496-4e7b-80fd-040abde4b2ba admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:38.905598 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30291037-d1e2-4772-ba8e-e03e141047f1 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:38.909505 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8117fb11-f78c-44ed-9e2b-acf599606137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:38.910723 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117fb11-f78c-44ed-9e2b-acf599606137 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:38.911213 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117fb11-f78c-44ed-9e2b-acf599606137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:38.911682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117fb11-f78c-44ed-9e2b-acf599606137 admin admin] PATH_INFO: `/v3/projects/81af60892d324c7ea1eb2b04440584b1/users/56142bbce9994044be921a231a51cae4/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:38.911975 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5bf70b-aa06-468e-9fda-18685c819095 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:38.913145 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5bf70b-aa06-468e-9fda-18685c819095 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:38.913829 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5bf70b-aa06-468e-9fda-18685c819095 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:38.914472 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5bf70b-aa06-468e-9fda-18685c819095 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:38.933348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8117fb11-f78c-44ed-9e2b-acf599606137 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81af60892d324c7ea1eb2b04440584b1, user_id=56142bbce9994044be921a231a51cae4, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:38.937247 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8117fb11-f78c-44ed-9e2b-acf599606137 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:38.959070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1129/2291] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:34:38 2022] PUT /identity/v3/projects/81af60892d324c7ea1eb2b04440584b1/users/56142bbce9994044be921a231a51cae4/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:34:38.965237 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8117fb11-f78c-44ed-9e2b-acf599606137 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:38.981375 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:34:39.029942 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5bf70b-aa06-468e-9fda-18685c819095 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:39.032426 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5bf70b-aa06-468e-9fda-18685c819095 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:39.048085 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1163/2292] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:34:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:34:39.169173 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8117fb11-f78c-44ed-9e2b-acf599606137 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:39.178923 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6452d30d-ee8c-4ce9-aefc-c8fb1f6a25f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:39.179842 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452d30d-ee8c-4ce9-aefc-c8fb1f6a25f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:39.180155 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452d30d-ee8c-4ce9-aefc-c8fb1f6a25f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:39.181034 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452d30d-ee8c-4ce9-aefc-c8fb1f6a25f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:39.182284 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6452d30d-ee8c-4ce9-aefc-c8fb1f6a25f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:39.197539 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6452d30d-ee8c-4ce9-aefc-c8fb1f6a25f0 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:39.228900 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1130/2293] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:34:39.234274 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5bf70b-aa06-468e-9fda-18685c819095 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:39.270903 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5bf70b-aa06-468e-9fda-18685c819095 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:39.287084 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361eae13-81ce-494e-9e19-f004425f1436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:39.289882 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361eae13-81ce-494e-9e19-f004425f1436 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:39.290181 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361eae13-81ce-494e-9e19-f004425f1436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:39.290492 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361eae13-81ce-494e-9e19-f004425f1436 admin admin] PATH_INFO: `/v3/projects/81af60892d324c7ea1eb2b04440584b1/users/56142bbce9994044be921a231a51cae4/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:39.292814 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361eae13-81ce-494e-9e19-f004425f1436 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81af60892d324c7ea1eb2b04440584b1, user_id=56142bbce9994044be921a231a51cae4, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:39.295812 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361eae13-81ce-494e-9e19-f004425f1436 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:39.311086 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1164/2294] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:34:39 2022] PUT /identity/v3/projects/81af60892d324c7ea1eb2b04440584b1/users/56142bbce9994044be921a231a51cae4/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:34:40.223962 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6452d30d-ee8c-4ce9-aefc-c8fb1f6a25f0 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:40.226807 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6452d30d-ee8c-4ce9-aefc-c8fb1f6a25f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:40.390505 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6452d30d-ee8c-4ce9-aefc-c8fb1f6a25f0 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:40.553396 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175878a-766f-46e1-8fa9-38449d2b07df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:40.554760 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175878a-766f-46e1-8fa9-38449d2b07df service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:40.555233 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175878a-766f-46e1-8fa9-38449d2b07df service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:40.555609 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175878a-766f-46e1-8fa9-38449d2b07df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:40.700410 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7175878a-766f-46e1-8fa9-38449d2b07df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:40.703641 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7175878a-766f-46e1-8fa9-38449d2b07df service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:40.726969 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1131/2295] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:34:40 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:34:42.481931 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361eae13-81ce-494e-9e19-f004425f1436 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:42.487357 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-361eae13-81ce-494e-9e19-f004425f1436 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:42.701846 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361eae13-81ce-494e-9e19-f004425f1436 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:42.708833 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9efd10-a3db-4132-8128-86ae0c9052eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:42.709775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9efd10-a3db-4132-8128-86ae0c9052eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:42.710046 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9efd10-a3db-4132-8128-86ae0c9052eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:42.710720 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9efd10-a3db-4132-8128-86ae0c9052eb admin admin] PATH_INFO: `/v3/users/591c0bf1ed2c401dbfd367dfff628b5f` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:42.712515 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9efd10-a3db-4132-8128-86ae0c9052eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=591c0bf1ed2c401dbfd367dfff628b5f)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:42.715028 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9efd10-a3db-4132-8128-86ae0c9052eb admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:42.758667 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:34:42.849496 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9a9efd10-a3db-4132-8128-86ae0c9052eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '591c0bf1ed2c401dbfd367dfff628b5f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:42.912126 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9a9efd10-a3db-4132-8128-86ae0c9052eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '591c0bf1ed2c401dbfd367dfff628b5f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:42.912923 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9a9efd10-a3db-4132-8128-86ae0c9052eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '591c0bf1ed2c401dbfd367dfff628b5f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:42.915104 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9a9efd10-a3db-4132-8128-86ae0c9052eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '591c0bf1ed2c401dbfd367dfff628b5f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:42.935095 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9a9efd10-a3db-4132-8128-86ae0c9052eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '591c0bf1ed2c401dbfd367dfff628b5f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:42.966865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1165/2296] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:34:42 2022] DELETE /identity/v3/users/591c0bf1ed2c401dbfd367dfff628b5f => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:34:42.970414 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415a82b-0533-43b0-a67b-2663d3c6beba None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:42.971150 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415a82b-0533-43b0-a67b-2663d3c6beba None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:42.971412 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415a82b-0533-43b0-a67b-2663d3c6beba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:43.001642 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:34:43.049426 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-7415a82b-0533-43b0-a67b-2663d3c6beba None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:34:43.055528 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7415a82b-0533-43b0-a67b-2663d3c6beba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:43.198520 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1132/2297] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:34:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:34:43.208636 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9efd10-a3db-4132-8128-86ae0c9052eb admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:43.213108 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9a9efd10-a3db-4132-8128-86ae0c9052eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:43.363584 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9efd10-a3db-4132-8128-86ae0c9052eb admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:43.371276 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a90b988-48f9-48df-8178-cca5aae92c08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:43.372345 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a90b988-48f9-48df-8178-cca5aae92c08 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:43.372640 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a90b988-48f9-48df-8178-cca5aae92c08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:43.372928 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a90b988-48f9-48df-8178-cca5aae92c08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:43.396392 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a90b988-48f9-48df-8178-cca5aae92c08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:43.398796 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a90b988-48f9-48df-8178-cca5aae92c08 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:43.423453 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1166/2298] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:34:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:34:43.982981 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7415a82b-0533-43b0-a67b-2663d3c6beba None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:43.988334 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7415a82b-0533-43b0-a67b-2663d3c6beba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:44.042015 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7415a82b-0533-43b0-a67b-2663d3c6beba None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:44.071563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b1cda-374d-4f42-bed3-74f71c5724a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:44.073531 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b1cda-374d-4f42-bed3-74f71c5724a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:44.074414 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b1cda-374d-4f42-bed3-74f71c5724a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:44.075146 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b1cda-374d-4f42-bed3-74f71c5724a9 admin admin] PATH_INFO: `/v3/projects/86b9e670ae6144548076173273026d17` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:44.082903 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1b1cda-374d-4f42-bed3-74f71c5724a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86b9e670ae6144548076173273026d17)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:44.088763 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1b1cda-374d-4f42-bed3-74f71c5724a9 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:44.288786 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-3d1b1cda-374d-4f42-bed3-74f71c5724a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86b9e670ae6144548076173273026d17', 'request_id': 'req-3d1b1cda-374d-4f42-bed3-74f71c5724a9'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:44.339313 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-3d1b1cda-374d-4f42-bed3-74f71c5724a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86b9e670ae6144548076173273026d17', 'request_id': 'req-3d1b1cda-374d-4f42-bed3-74f71c5724a9'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:44.367801 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-3d1b1cda-374d-4f42-bed3-74f71c5724a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86b9e670ae6144548076173273026d17', 'request_id': 'req-3d1b1cda-374d-4f42-bed3-74f71c5724a9'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:44.415836 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-3d1b1cda-374d-4f42-bed3-74f71c5724a9 admin admin] The token cache is being invalidate because project 86b9e670ae6144548076173273026d17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:34:44.416131 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-3d1b1cda-374d-4f42-bed3-74f71c5724a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:44.426435 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-3d1b1cda-374d-4f42-bed3-74f71c5724a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:44.438643 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1133/2299] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:34:43 2022] DELETE /identity/v3/projects/86b9e670ae6144548076173273026d17 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:34:44.443501 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008e37e-25a5-4a42-a1a6-347d4a582bff None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:44.444347 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008e37e-25a5-4a42-a1a6-347d4a582bff None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:44.444589 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008e37e-25a5-4a42-a1a6-347d4a582bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:44.514555 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0008e37e-25a5-4a42-a1a6-347d4a582bff None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:34:44.516881 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0008e37e-25a5-4a42-a1a6-347d4a582bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:44.863315 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1167/2300] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:34:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:34:44.877553 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51f4c0-6953-4f2c-b0fc-16955645ed50 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:44.879591 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51f4c0-6953-4f2c-b0fc-16955645ed50 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:44.881589 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51f4c0-6953-4f2c-b0fc-16955645ed50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:44.955708 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-9c51f4c0-6953-4f2c-b0fc-16955645ed50 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:34:44.964644 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9c51f4c0-6953-4f2c-b0fc-16955645ed50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:45.001657 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1134/2301] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:34:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:34:45.007885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008e37e-25a5-4a42-a1a6-347d4a582bff None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:45.038445 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008e37e-25a5-4a42-a1a6-347d4a582bff None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:45.059804 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b7facc-c252-4405-b0bc-fdbe6abb2df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:45.061919 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7facc-c252-4405-b0bc-fdbe6abb2df4 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:45.062981 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7facc-c252-4405-b0bc-fdbe6abb2df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:45.064711 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7facc-c252-4405-b0bc-fdbe6abb2df4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:45.067979 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b7facc-c252-4405-b0bc-fdbe6abb2df4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:34:45.068515 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b7facc-c252-4405-b0bc-fdbe6abb2df4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:45.078615 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b7facc-c252-4405-b0bc-fdbe6abb2df4 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:45.134222 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1168/2302] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:34:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:34:45.141023 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51f4c0-6953-4f2c-b0fc-16955645ed50 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:45.174483 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51f4c0-6953-4f2c-b0fc-16955645ed50 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:45.190213 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf3c9d-d63f-42a9-920f-e0fd33a679e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:45.191375 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf3c9d-d63f-42a9-920f-e0fd33a679e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:45.191906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf3c9d-d63f-42a9-920f-e0fd33a679e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:45.192753 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf3c9d-d63f-42a9-920f-e0fd33a679e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:45.193801 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf3c9d-d63f-42a9-920f-e0fd33a679e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:45.196928 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf3c9d-d63f-42a9-920f-e0fd33a679e1 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:45.253999 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1135/2303] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:34:45 2022] POST /identity/v3/projects => generated 397 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:34:45.263067 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b7facc-c252-4405-b0bc-fdbe6abb2df4 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:45.315771 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b7facc-c252-4405-b0bc-fdbe6abb2df4 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:45.442650 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:34:45.573273 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71511263-db10-48cf-ae03-106f24ae4bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:45.579013 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71511263-db10-48cf-ae03-106f24ae4bce admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:45.579818 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71511263-db10-48cf-ae03-106f24ae4bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:45.580537 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71511263-db10-48cf-ae03-106f24ae4bce admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:45.594330 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71511263-db10-48cf-ae03-106f24ae4bce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:45.596313 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71511263-db10-48cf-ae03-106f24ae4bce admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:45.669550 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1169/2304] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:34:45 2022] POST /identity/v3/users => generated 299 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:34:45.675934 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf3c9d-d63f-42a9-920f-e0fd33a679e1 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:45.696892 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf3c9d-d63f-42a9-920f-e0fd33a679e1 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:45.709777 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f64d5ea-2a8e-486b-a02f-ec246a70f7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:45.710948 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64d5ea-2a8e-486b-a02f-ec246a70f7f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:45.711734 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64d5ea-2a8e-486b-a02f-ec246a70f7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:45.712363 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64d5ea-2a8e-486b-a02f-ec246a70f7f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:45.713365 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f64d5ea-2a8e-486b-a02f-ec246a70f7f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:45.718516 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f64d5ea-2a8e-486b-a02f-ec246a70f7f6 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:45.751967 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1136/2305] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:34:45.757660 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71511263-db10-48cf-ae03-106f24ae4bce admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:45.785847 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71511263-db10-48cf-ae03-106f24ae4bce admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:45.804810 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993a0930-3aca-4357-9970-0705c0fc862a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:45.805934 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993a0930-3aca-4357-9970-0705c0fc862a admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:45.806469 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993a0930-3aca-4357-9970-0705c0fc862a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:45.807046 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993a0930-3aca-4357-9970-0705c0fc862a admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:45.809518 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993a0930-3aca-4357-9970-0705c0fc862a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:45.811639 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993a0930-3aca-4357-9970-0705c0fc862a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:45.832708 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1170/2306] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 21:34:45.839321 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f64d5ea-2a8e-486b-a02f-ec246a70f7f6 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:45.870208 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f64d5ea-2a8e-486b-a02f-ec246a70f7f6 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:45.889818 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f639c35a-7bf9-4e05-89f4-4985f2e9c174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:45.894721 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639c35a-7bf9-4e05-89f4-4985f2e9c174 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:45.895290 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639c35a-7bf9-4e05-89f4-4985f2e9c174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:45.895727 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639c35a-7bf9-4e05-89f4-4985f2e9c174 admin admin] PATH_INFO: `/v3/projects/9cbd1aaecb1f4f608d69419aa429195c/users/006bbd4ab3b94098afb08fa93c6c3b51/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:45.945819 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f639c35a-7bf9-4e05-89f4-4985f2e9c174 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cbd1aaecb1f4f608d69419aa429195c, user_id=006bbd4ab3b94098afb08fa93c6c3b51, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:45.951135 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f639c35a-7bf9-4e05-89f4-4985f2e9c174 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:45.993293 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1137/2307] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:34:45 2022] PUT /identity/v3/projects/9cbd1aaecb1f4f608d69419aa429195c/users/006bbd4ab3b94098afb08fa93c6c3b51/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:34:46.013208 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65920e-4836-4407-8c79-349fb354c359 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:46.013825 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65920e-4836-4407-8c79-349fb354c359 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:46.014409 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65920e-4836-4407-8c79-349fb354c359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:46.076879 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-7a65920e-4836-4407-8c79-349fb354c359 None None] MFA Rules not processed for user `006bbd4ab3b94098afb08fa93c6c3b51`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:34:46.080854 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7a65920e-4836-4407-8c79-349fb354c359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:46.295025 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1171/2308] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:34:46 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:34:46.309090 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f639c35a-7bf9-4e05-89f4-4985f2e9c174 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:46.312249 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f639c35a-7bf9-4e05-89f4-4985f2e9c174 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:46.393306 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:34:46.460279 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f639c35a-7bf9-4e05-89f4-4985f2e9c174 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:46.467826 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e36848-0621-4bfa-bd3e-ac5c3cb5d7d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:46.468799 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e36848-0621-4bfa-bd3e-ac5c3cb5d7d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:46.469194 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e36848-0621-4bfa-bd3e-ac5c3cb5d7d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:46.469595 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e36848-0621-4bfa-bd3e-ac5c3cb5d7d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:46.494358 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e36848-0621-4bfa-bd3e-ac5c3cb5d7d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:46.496206 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e36848-0621-4bfa-bd3e-ac5c3cb5d7d5 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:46.531762 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1138/2309] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:34:46 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:34:47.400654 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a65920e-4836-4407-8c79-349fb354c359 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:47.402762 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7a65920e-4836-4407-8c79-349fb354c359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:47.547152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a65920e-4836-4407-8c79-349fb354c359 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:47.553363 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f869ff2-3b14-4848-9a87-031fa5ca29e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:47.554184 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f869ff2-3b14-4848-9a87-031fa5ca29e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:47.555390 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f869ff2-3b14-4848-9a87-031fa5ca29e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:47.555911 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f869ff2-3b14-4848-9a87-031fa5ca29e4 admin admin] PATH_INFO: `/v3/users/3346f90be94247a2bdd8b354e631bd16` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:47.559950 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f869ff2-3b14-4848-9a87-031fa5ca29e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3346f90be94247a2bdd8b354e631bd16)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:47.571561 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f869ff2-3b14-4848-9a87-031fa5ca29e4 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:47.705482 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:34:47.708382 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9f869ff2-3b14-4848-9a87-031fa5ca29e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3346f90be94247a2bdd8b354e631bd16'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:47.749704 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9f869ff2-3b14-4848-9a87-031fa5ca29e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3346f90be94247a2bdd8b354e631bd16'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:47.751792 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9f869ff2-3b14-4848-9a87-031fa5ca29e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3346f90be94247a2bdd8b354e631bd16'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:47.753158 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9f869ff2-3b14-4848-9a87-031fa5ca29e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3346f90be94247a2bdd8b354e631bd16'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:47.769588 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9f869ff2-3b14-4848-9a87-031fa5ca29e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3346f90be94247a2bdd8b354e631bd16'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:47.793830 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1172/2310] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:34:47 2022] DELETE /identity/v3/users/3346f90be94247a2bdd8b354e631bd16 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:34:47.799650 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550db7e-a991-45b2-b118-4c2e8b5a75cb None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:47.800612 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550db7e-a991-45b2-b118-4c2e8b5a75cb None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:47.800901 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550db7e-a991-45b2-b118-4c2e8b5a75cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:47.828053 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-c550db7e-a991-45b2-b118-4c2e8b5a75cb None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:34:47.829797 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c550db7e-a991-45b2-b118-4c2e8b5a75cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:47.998495 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1139/2311] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:34:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:34:48.013364 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f869ff2-3b14-4848-9a87-031fa5ca29e4 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:48.016942 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9f869ff2-3b14-4848-9a87-031fa5ca29e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:48.116658 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f869ff2-3b14-4848-9a87-031fa5ca29e4 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:48.121495 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a6524-528c-4c82-901f-ae823f4babaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:48.122195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a6524-528c-4c82-901f-ae823f4babaa service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:48.123286 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a6524-528c-4c82-901f-ae823f4babaa service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:48.123619 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a6524-528c-4c82-901f-ae823f4babaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:48.138416 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a6524-528c-4c82-901f-ae823f4babaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:48.140960 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a6524-528c-4c82-901f-ae823f4babaa service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:48.157937 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1173/2312] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:34:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:34:48.656910 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133f32a-4688-4dce-a014-2238908a710e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:48.657551 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133f32a-4688-4dce-a014-2238908a710e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:48.658710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133f32a-4688-4dce-a014-2238908a710e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:48.710483 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b133f32a-4688-4dce-a014-2238908a710e None None] MFA Rules not processed for user `56142bbce9994044be921a231a51cae4`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:34:48.712244 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b133f32a-4688-4dce-a014-2238908a710e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:48.739629 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e1cd2e-95d7-4804-8bd9-4257d90e8d60 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:48.740158 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e1cd2e-95d7-4804-8bd9-4257d90e8d60 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:48.740158 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e1cd2e-95d7-4804-8bd9-4257d90e8d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:48.760631 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f0e1cd2e-95d7-4804-8bd9-4257d90e8d60 None None] MFA Rules not processed for user `1589968f375946a38e82ec484afc837f`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:34:48.761849 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f0e1cd2e-95d7-4804-8bd9-4257d90e8d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:48.856040 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1174/2313] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:34:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:34:48.858765 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e1cd2e-95d7-4804-8bd9-4257d90e8d60 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:48.869649 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f0e1cd2e-95d7-4804-8bd9-4257d90e8d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:48.892669 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1140/2314] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:34:48 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:34:48.893484 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e1cd2e-95d7-4804-8bd9-4257d90e8d60 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:48.903074 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2a36c9-45d1-4e61-9793-e182194bf5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:48.904148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a36c9-45d1-4e61-9793-e182194bf5d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:48.904536 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a36c9-45d1-4e61-9793-e182194bf5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:48.905012 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a36c9-45d1-4e61-9793-e182194bf5d8 admin admin] PATH_INFO: `/v3/projects/60ba2171e8a2449bbf59e191b261ac57` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:48.907100 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2a36c9-45d1-4e61-9793-e182194bf5d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60ba2171e8a2449bbf59e191b261ac57)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:48.914296 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133f32a-4688-4dce-a014-2238908a710e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:48.917527 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b133f32a-4688-4dce-a014-2238908a710e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:48.918082 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2a36c9-45d1-4e61-9793-e182194bf5d8 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:49.008812 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9d2a36c9-45d1-4e61-9793-e182194bf5d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60ba2171e8a2449bbf59e191b261ac57', 'request_id': 'req-9d2a36c9-45d1-4e61-9793-e182194bf5d8'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:49.017621 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9d2a36c9-45d1-4e61-9793-e182194bf5d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60ba2171e8a2449bbf59e191b261ac57', 'request_id': 'req-9d2a36c9-45d1-4e61-9793-e182194bf5d8'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:49.025946 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9d2a36c9-45d1-4e61-9793-e182194bf5d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60ba2171e8a2449bbf59e191b261ac57', 'request_id': 'req-9d2a36c9-45d1-4e61-9793-e182194bf5d8'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:49.028716 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:34:49.036644 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:34:49.057782 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9d2a36c9-45d1-4e61-9793-e182194bf5d8 admin admin] The token cache is being invalidate because project 60ba2171e8a2449bbf59e191b261ac57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:34:49.058127 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9d2a36c9-45d1-4e61-9793-e182194bf5d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:49.059679 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9d2a36c9-45d1-4e61-9793-e182194bf5d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:49.063233 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1175/2315] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:34:48 2022] DELETE /identity/v3/projects/60ba2171e8a2449bbf59e191b261ac57 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:34:49.076067 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2a36c9-45d1-4e61-9793-e182194bf5d8 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:49.079637 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9d2a36c9-45d1-4e61-9793-e182194bf5d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:49.101883 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133f32a-4688-4dce-a014-2238908a710e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:49.162033 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2a36c9-45d1-4e61-9793-e182194bf5d8 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:49.167893 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce61e537-4714-439e-9a7c-6af290c071d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:49.169099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61e537-4714-439e-9a7c-6af290c071d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:49.169491 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61e537-4714-439e-9a7c-6af290c071d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:49.169913 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61e537-4714-439e-9a7c-6af290c071d0 admin admin] PATH_INFO: `/v3/users/8a665e53d92a4740961253eacda24b55` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:49.172000 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce61e537-4714-439e-9a7c-6af290c071d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a665e53d92a4740961253eacda24b55)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:49.174272 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce61e537-4714-439e-9a7c-6af290c071d0 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:49.215923 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5346bf-a7e4-4b77-944b-ae4914b36d39 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea874ca4088d4e6c83c9d0497323cf6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:49.220073 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5346bf-a7e4-4b77-944b-ae4914b36d39 service glance] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:49.220526 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5346bf-a7e4-4b77-944b-ae4914b36d39 service glance] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:49.220905 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5346bf-a7e4-4b77-944b-ae4914b36d39 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:49.225452 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0d5346bf-a7e4-4b77-944b-ae4914b36d39 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:49.242632 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ce61e537-4714-439e-9a7c-6af290c071d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a665e53d92a4740961253eacda24b55'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:49.267069 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ce61e537-4714-439e-9a7c-6af290c071d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a665e53d92a4740961253eacda24b55'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:49.268212 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ce61e537-4714-439e-9a7c-6af290c071d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a665e53d92a4740961253eacda24b55'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:49.269122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ce61e537-4714-439e-9a7c-6af290c071d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a665e53d92a4740961253eacda24b55'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:49.281450 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ce61e537-4714-439e-9a7c-6af290c071d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a665e53d92a4740961253eacda24b55'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:49.310482 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1176/2316] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:34:49 2022] DELETE /identity/v3/users/8a665e53d92a4740961253eacda24b55 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:34:49.318412 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5346bf-a7e4-4b77-944b-ae4914b36d39 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:49.319967 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5346bf-a7e4-4b77-944b-ae4914b36d39 service glance] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:49.332535 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1141/2317] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:34:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:34:49.544471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7e4fe-ae20-4d91-b25c-57bf2c3e294f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:49.544754 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7e4fe-ae20-4d91-b25c-57bf2c3e294f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:49.545037 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7e4fe-ae20-4d91-b25c-57bf2c3e294f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:49.665521 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f0f7e4fe-ae20-4d91-b25c-57bf2c3e294f None None] MFA Rules not processed for user `eba91dff7cac401e81f6fb6391874af1`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:34:49.667007 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f0f7e4fe-ae20-4d91-b25c-57bf2c3e294f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:49.734527 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5346bf-a7e4-4b77-944b-ae4914b36d39 service glance] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:49.739517 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:34:49.767962 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5346bf-a7e4-4b77-944b-ae4914b36d39 service glance] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:49.834728 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1177/2318] 10.222.0.65 () {58 vars in 1009 bytes} [Thu May 19 21:34:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:34:49.886324 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce2b272-925c-4059-b8cf-c277af797775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:49.886581 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2b272-925c-4059-b8cf-c277af797775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:49.886868 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2b272-925c-4059-b8cf-c277af797775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:49.887144 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2b272-925c-4059-b8cf-c277af797775 admin admin] PATH_INFO: `/v3/projects/5bb4fd090a8c40aab3a565a6d1596910` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:49.890024 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce2b272-925c-4059-b8cf-c277af797775 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bb4fd090a8c40aab3a565a6d1596910)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:49.892300 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce2b272-925c-4059-b8cf-c277af797775 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:49.897488 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7e4fe-ae20-4d91-b25c-57bf2c3e294f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:49.899105 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f0f7e4fe-ae20-4d91-b25c-57bf2c3e294f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:49.970921 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:34:49.980423 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bce2b272-925c-4059-b8cf-c277af797775 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bb4fd090a8c40aab3a565a6d1596910', 'request_id': 'req-bce2b272-925c-4059-b8cf-c277af797775'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:49.991148 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bce2b272-925c-4059-b8cf-c277af797775 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bb4fd090a8c40aab3a565a6d1596910', 'request_id': 'req-bce2b272-925c-4059-b8cf-c277af797775'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:50.003136 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bce2b272-925c-4059-b8cf-c277af797775 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bb4fd090a8c40aab3a565a6d1596910', 'request_id': 'req-bce2b272-925c-4059-b8cf-c277af797775'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:50.022782 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bce2b272-925c-4059-b8cf-c277af797775 admin admin] The token cache is being invalidate because project 5bb4fd090a8c40aab3a565a6d1596910 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:34:50.023454 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bce2b272-925c-4059-b8cf-c277af797775 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:50.024692 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bce2b272-925c-4059-b8cf-c277af797775 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:50.026275 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1142/2319] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:34:49 2022] DELETE /identity/v3/projects/5bb4fd090a8c40aab3a565a6d1596910 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:34:50.030564 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce2b272-925c-4059-b8cf-c277af797775 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:50.032459 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-bce2b272-925c-4059-b8cf-c277af797775 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:50.090191 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7e4fe-ae20-4d91-b25c-57bf2c3e294f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:50.242449 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce2b272-925c-4059-b8cf-c277af797775 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:50.253443 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230e7dc-0c6e-4429-8e07-701c1f5ada84 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e07984b36934aff9052541ef2a56f36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:50.254725 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230e7dc-0c6e-4429-8e07-701c1f5ada84 service swift] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:50.255546 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230e7dc-0c6e-4429-8e07-701c1f5ada84 service swift] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:50.255972 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230e7dc-0c6e-4429-8e07-701c1f5ada84 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:50.258925 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2230e7dc-0c6e-4429-8e07-701c1f5ada84 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:50.259299 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0701aaf1-4e6f-4b81-8fb6-31766c823bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:50.261459 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701aaf1-4e6f-4b81-8fb6-31766c823bb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:50.262173 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701aaf1-4e6f-4b81-8fb6-31766c823bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:50.263040 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701aaf1-4e6f-4b81-8fb6-31766c823bb0 admin admin] PATH_INFO: `/v3/users/f434028bee6b4d328cbf1ce06a857c55` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:50.265769 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0701aaf1-4e6f-4b81-8fb6-31766c823bb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f434028bee6b4d328cbf1ce06a857c55)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:50.271198 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0701aaf1-4e6f-4b81-8fb6-31766c823bb0 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:50.356749 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0701aaf1-4e6f-4b81-8fb6-31766c823bb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f434028bee6b4d328cbf1ce06a857c55'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:50.358191 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0701aaf1-4e6f-4b81-8fb6-31766c823bb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f434028bee6b4d328cbf1ce06a857c55'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:50.384981 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:34:50.385526 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0701aaf1-4e6f-4b81-8fb6-31766c823bb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f434028bee6b4d328cbf1ce06a857c55'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:50.399703 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0701aaf1-4e6f-4b81-8fb6-31766c823bb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f434028bee6b4d328cbf1ce06a857c55'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:50.407175 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2230e7dc-0c6e-4429-8e07-701c1f5ada84 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:50.413228 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2230e7dc-0c6e-4429-8e07-701c1f5ada84 service swift] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:50.424223 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0701aaf1-4e6f-4b81-8fb6-31766c823bb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f434028bee6b4d328cbf1ce06a857c55'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:50.431408 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1178/2320] 10.222.0.65 () {60 vars in 1384 bytes} [Thu May 19 21:34:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 19 21:34:50.433381 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1143/2321] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:34:50 2022] DELETE /identity/v3/users/f434028bee6b4d328cbf1ce06a857c55 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:34:51.140444 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230e7dc-0c6e-4429-8e07-701c1f5ada84 service swift] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:51.155992 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230e7dc-0c6e-4429-8e07-701c1f5ada84 service swift] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:51.261565 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72572be0-7668-40d8-aaf7-e5514391fd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:51.262375 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72572be0-7668-40d8-aaf7-e5514391fd34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:51.262920 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72572be0-7668-40d8-aaf7-e5514391fd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:51.263317 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72572be0-7668-40d8-aaf7-e5514391fd34 admin admin] PATH_INFO: `/v3/projects/009fdb22845142b690966249f44116f2` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:51.265358 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72572be0-7668-40d8-aaf7-e5514391fd34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=009fdb22845142b690966249f44116f2)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:51.267605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72572be0-7668-40d8-aaf7-e5514391fd34 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:51.385996 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-72572be0-7668-40d8-aaf7-e5514391fd34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '009fdb22845142b690966249f44116f2', 'request_id': 'req-72572be0-7668-40d8-aaf7-e5514391fd34'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:51.401372 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-72572be0-7668-40d8-aaf7-e5514391fd34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '009fdb22845142b690966249f44116f2', 'request_id': 'req-72572be0-7668-40d8-aaf7-e5514391fd34'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:51.440213 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-72572be0-7668-40d8-aaf7-e5514391fd34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '009fdb22845142b690966249f44116f2', 'request_id': 'req-72572be0-7668-40d8-aaf7-e5514391fd34'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:51.469659 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-72572be0-7668-40d8-aaf7-e5514391fd34 admin admin] The token cache is being invalidate because project 009fdb22845142b690966249f44116f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:34:51.470134 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-72572be0-7668-40d8-aaf7-e5514391fd34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:51.471092 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-72572be0-7668-40d8-aaf7-e5514391fd34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:34:51.474358 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1179/2322] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:34:51 2022] DELETE /identity/v3/projects/009fdb22845142b690966249f44116f2 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:34:51.484579 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61829f7-e96e-405f-85eb-84b4a7587ce3 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:51.484579 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61829f7-e96e-405f-85eb-84b4a7587ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:51.484579 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61829f7-e96e-405f-85eb-84b4a7587ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:51.606732 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b61829f7-e96e-405f-85eb-84b4a7587ce3 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:34:51.608994 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b61829f7-e96e-405f-85eb-84b4a7587ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:51.718502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:34:51.766376 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1144/2323] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:34:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:34:51.780394 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239dfdc-c3ab-4251-9b95-d7dd9b002b0c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:51.780898 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239dfdc-c3ab-4251-9b95-d7dd9b002b0c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:51.781441 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239dfdc-c3ab-4251-9b95-d7dd9b002b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:51.915054 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-2239dfdc-c3ab-4251-9b95-d7dd9b002b0c None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:34:51.918875 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2239dfdc-c3ab-4251-9b95-d7dd9b002b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:51.952109 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61829f7-e96e-405f-85eb-84b4a7587ce3 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:51.952569 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1180/2324] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:34:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:34:51.969858 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61829f7-e96e-405f-85eb-84b4a7587ce3 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:51.990355 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710636ae-6b5a-4529-bde4-e65747ba36e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:51.991134 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710636ae-6b5a-4529-bde4-e65747ba36e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:51.991500 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710636ae-6b5a-4529-bde4-e65747ba36e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:51.991686 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710636ae-6b5a-4529-bde4-e65747ba36e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:51.992401 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710636ae-6b5a-4529-bde4-e65747ba36e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:34:51.992709 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710636ae-6b5a-4529-bde4-e65747ba36e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:51.994508 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710636ae-6b5a-4529-bde4-e65747ba36e6 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:52.026308 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1145/2325] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:34:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:34:52.032108 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239dfdc-c3ab-4251-9b95-d7dd9b002b0c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:52.046069 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239dfdc-c3ab-4251-9b95-d7dd9b002b0c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:52.053414 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0212cf-1da6-4086-b701-932b1c233d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:52.055411 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0212cf-1da6-4086-b701-932b1c233d23 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:52.055411 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0212cf-1da6-4086-b701-932b1c233d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:52.055411 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0212cf-1da6-4086-b701-932b1c233d23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:52.055797 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0212cf-1da6-4086-b701-932b1c233d23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:52.057606 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0212cf-1da6-4086-b701-932b1c233d23 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:52.115420 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710636ae-6b5a-4529-bde4-e65747ba36e6 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:52.128917 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1181/2326] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:34:52 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:34:52.140567 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710636ae-6b5a-4529-bde4-e65747ba36e6 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:52.244516 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f40233-ce46-4b2c-acca-018b48f1fcc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:52.245193 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f40233-ce46-4b2c-acca-018b48f1fcc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:52.245525 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f40233-ce46-4b2c-acca-018b48f1fcc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:52.245954 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f40233-ce46-4b2c-acca-018b48f1fcc0 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:52.246747 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f40233-ce46-4b2c-acca-018b48f1fcc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:52.249263 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f40233-ce46-4b2c-acca-018b48f1fcc0 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:52.286843 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0212cf-1da6-4086-b701-932b1c233d23 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:52.287770 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1146/2327] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:34:52 2022] POST /identity/v3/users => generated 288 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:34:52.300834 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0212cf-1da6-4086-b701-932b1c233d23 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:52.310381 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193cb7c7-4ab5-4821-83bc-c61c770af985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:52.311854 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193cb7c7-4ab5-4821-83bc-c61c770af985 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:52.312535 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193cb7c7-4ab5-4821-83bc-c61c770af985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:52.313247 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193cb7c7-4ab5-4821-83bc-c61c770af985 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:52.317802 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193cb7c7-4ab5-4821-83bc-c61c770af985 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:52.322436 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193cb7c7-4ab5-4821-83bc-c61c770af985 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:52.348149 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1182/2328] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:34:52 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 21:34:52.352483 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f40233-ce46-4b2c-acca-018b48f1fcc0 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:52.371653 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f40233-ce46-4b2c-acca-018b48f1fcc0 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:52.388434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f948491-ec4f-4719-973b-1b04d87c2b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:52.389685 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f948491-ec4f-4719-973b-1b04d87c2b46 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:52.389685 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f948491-ec4f-4719-973b-1b04d87c2b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:52.389953 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f948491-ec4f-4719-973b-1b04d87c2b46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:52.390573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f948491-ec4f-4719-973b-1b04d87c2b46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:52.392330 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f948491-ec4f-4719-973b-1b04d87c2b46 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:52.449326 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193cb7c7-4ab5-4821-83bc-c61c770af985 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:52.452171 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1147/2329] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:34:52 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:34:52.470634 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193cb7c7-4ab5-4821-83bc-c61c770af985 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:52.493897 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbff05f-771b-4af6-a6b5-938c7e59de35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:52.496648 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbff05f-771b-4af6-a6b5-938c7e59de35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:52.497799 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbff05f-771b-4af6-a6b5-938c7e59de35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:52.499498 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbff05f-771b-4af6-a6b5-938c7e59de35 admin admin] PATH_INFO: `/v3/projects/268378e6b7704c0882ba3967414051d8/users/5fd88c8a7f6d49e8bb0d0045183c1beb/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:52.546809 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbff05f-771b-4af6-a6b5-938c7e59de35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=268378e6b7704c0882ba3967414051d8, user_id=5fd88c8a7f6d49e8bb0d0045183c1beb, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:52.554916 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbff05f-771b-4af6-a6b5-938c7e59de35 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:52.612387 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1183/2330] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:34:52 2022] PUT /identity/v3/projects/268378e6b7704c0882ba3967414051d8/users/5fd88c8a7f6d49e8bb0d0045183c1beb/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:34:52.624770 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad00f9-3333-44a6-a0cc-7ded9d05bd6b None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:52.624770 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad00f9-3333-44a6-a0cc-7ded9d05bd6b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:52.625455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad00f9-3333-44a6-a0cc-7ded9d05bd6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:52.660167 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-9fad00f9-3333-44a6-a0cc-7ded9d05bd6b None None] MFA Rules not processed for user `5fd88c8a7f6d49e8bb0d0045183c1beb`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:34:52.661477 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9fad00f9-3333-44a6-a0cc-7ded9d05bd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:52.897837 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1148/2331] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:34:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:34:52.913378 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbff05f-771b-4af6-a6b5-938c7e59de35 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:52.915410 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-dbbff05f-771b-4af6-a6b5-938c7e59de35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:52.929482 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0f2bc-d139-4951-ad16-c4caaf11450c None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:52.929946 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0f2bc-d139-4951-ad16-c4caaf11450c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:52.930432 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0f2bc-d139-4951-ad16-c4caaf11450c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:52.980218 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-fdc0f2bc-d139-4951-ad16-c4caaf11450c None None] MFA Rules not processed for user `eba91dff7cac401e81f6fb6391874af1`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:34:52.982160 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-fdc0f2bc-d139-4951-ad16-c4caaf11450c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:53.012709 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:34:53.117032 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbff05f-771b-4af6-a6b5-938c7e59de35 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:53.127053 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f5a72-c77a-47df-9029-607d882a9604 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:53.128007 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f5a72-c77a-47df-9029-607d882a9604 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:53.128869 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f5a72-c77a-47df-9029-607d882a9604 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:53.129534 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f5a72-c77a-47df-9029-607d882a9604 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:53.157004 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f5a72-c77a-47df-9029-607d882a9604 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:53.158689 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f5a72-c77a-47df-9029-607d882a9604 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:53.174047 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:34:53.184187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1184/2332] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:34:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:34:53.193010 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1149/2333] 10.222.0.65 () {58 vars in 1009 bytes} [Thu May 19 21:34:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:34:53.259207 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f5a72-c77a-47df-9029-607d882a9604 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:34:53.261842 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-587f5a72-c77a-47df-9029-607d882a9604 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:53.448166 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f5a72-c77a-47df-9029-607d882a9604 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:34:53.456631 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2a5397-2e1c-480c-924a-02bfaf9c98a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e07984b36934aff9052541ef2a56f36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:34:53.458059 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a5397-2e1c-480c-924a-02bfaf9c98a7 service swift] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:53.458495 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a5397-2e1c-480c-924a-02bfaf9c98a7 service swift] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:53.459235 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a5397-2e1c-480c-924a-02bfaf9c98a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:53.479676 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2a5397-2e1c-480c-924a-02bfaf9c98a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:34:53.486051 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2a5397-2e1c-480c-924a-02bfaf9c98a7 service swift] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:34:53.507109 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1185/2334] 10.222.0.65 () {60 vars in 1384 bytes} [Thu May 19 21:34:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 21:34:53.953168 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e433197-8657-4320-bd1d-a28797bd1f9e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:34:53.955999 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e433197-8657-4320-bd1d-a28797bd1f9e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:34:53.956977 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e433197-8657-4320-bd1d-a28797bd1f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:34:53.995379 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-3e433197-8657-4320-bd1d-a28797bd1f9e None None] MFA Rules not processed for user `0a58f82117ab437cab5607c204c86547`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:34:53.997029 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3e433197-8657-4320-bd1d-a28797bd1f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:34:54.105183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1150/2335] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:34:53 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:35:03.135084 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2a5397-2e1c-480c-924a-02bfaf9c98a7 service swift] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:35:03.136608 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fd2a5397-2e1c-480c-924a-02bfaf9c98a7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:35:03.302846 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2a5397-2e1c-480c-924a-02bfaf9c98a7 service swift] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:35:03.317771 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41510fc1-5cee-4583-8b3f-b1454ac1c9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:35:03.318866 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41510fc1-5cee-4583-8b3f-b1454ac1c9a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:03.319379 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41510fc1-5cee-4583-8b3f-b1454ac1c9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:03.319717 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41510fc1-5cee-4583-8b3f-b1454ac1c9a3 admin admin] PATH_INFO: `/v3/users/006bbd4ab3b94098afb08fa93c6c3b51` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:03.322890 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41510fc1-5cee-4583-8b3f-b1454ac1c9a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=006bbd4ab3b94098afb08fa93c6c3b51)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:35:03.324934 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41510fc1-5cee-4583-8b3f-b1454ac1c9a3 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:35:03.479057 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-41510fc1-5cee-4583-8b3f-b1454ac1c9a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '006bbd4ab3b94098afb08fa93c6c3b51'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:35:03.492870 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:35:03.521292 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-41510fc1-5cee-4583-8b3f-b1454ac1c9a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '006bbd4ab3b94098afb08fa93c6c3b51'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:35:03.522108 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-41510fc1-5cee-4583-8b3f-b1454ac1c9a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '006bbd4ab3b94098afb08fa93c6c3b51'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:35:03.522764 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-41510fc1-5cee-4583-8b3f-b1454ac1c9a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '006bbd4ab3b94098afb08fa93c6c3b51'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:35:03.551791 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-41510fc1-5cee-4583-8b3f-b1454ac1c9a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '006bbd4ab3b94098afb08fa93c6c3b51'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:35:03.602125 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1186/2336] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:35:03 2022] DELETE /identity/v3/users/006bbd4ab3b94098afb08fa93c6c3b51 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:35:03.607112 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca1b8e-07b4-40f3-a639-4825d2b7bdc5 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:03.608338 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca1b8e-07b4-40f3-a639-4825d2b7bdc5 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:03.608818 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca1b8e-07b4-40f3-a639-4825d2b7bdc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:03.645836 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-e5ca1b8e-07b4-40f3-a639-4825d2b7bdc5 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:35:03.648200 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e5ca1b8e-07b4-40f3-a639-4825d2b7bdc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:35:03.811947 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1151/2337] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:35:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:35:03.829007 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41510fc1-5cee-4583-8b3f-b1454ac1c9a3 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:35:03.830516 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-41510fc1-5cee-4583-8b3f-b1454ac1c9a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:35:03.952680 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41510fc1-5cee-4583-8b3f-b1454ac1c9a3 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:35:03.963106 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6eb60c-f1cb-4bea-a446-a4053995c68a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:35:03.963903 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6eb60c-f1cb-4bea-a446-a4053995c68a service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:03.964264 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6eb60c-f1cb-4bea-a446-a4053995c68a service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:03.964618 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6eb60c-f1cb-4bea-a446-a4053995c68a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:04.012000 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6eb60c-f1cb-4bea-a446-a4053995c68a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:35:04.015668 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6eb60c-f1cb-4bea-a446-a4053995c68a service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:35:04.038028 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1187/2338] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:35:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:35:05.253977 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca1b8e-07b4-40f3-a639-4825d2b7bdc5 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:35:05.256141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e5ca1b8e-07b4-40f3-a639-4825d2b7bdc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:35:05.302117 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca1b8e-07b4-40f3-a639-4825d2b7bdc5 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:35:05.310450 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b6543-df03-4284-a5ec-ccd9d3afa2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:35:05.311739 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b6543-df03-4284-a5ec-ccd9d3afa2fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:05.312163 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b6543-df03-4284-a5ec-ccd9d3afa2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:05.312538 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b6543-df03-4284-a5ec-ccd9d3afa2fb admin admin] PATH_INFO: `/v3/projects/9cbd1aaecb1f4f608d69419aa429195c` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:05.314164 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494b6543-df03-4284-a5ec-ccd9d3afa2fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cbd1aaecb1f4f608d69419aa429195c)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:35:05.316315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494b6543-df03-4284-a5ec-ccd9d3afa2fb admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:35:05.443357 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-494b6543-df03-4284-a5ec-ccd9d3afa2fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cbd1aaecb1f4f608d69419aa429195c', 'request_id': 'req-494b6543-df03-4284-a5ec-ccd9d3afa2fb'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:35:05.453790 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-494b6543-df03-4284-a5ec-ccd9d3afa2fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cbd1aaecb1f4f608d69419aa429195c', 'request_id': 'req-494b6543-df03-4284-a5ec-ccd9d3afa2fb'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:35:05.474371 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-494b6543-df03-4284-a5ec-ccd9d3afa2fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cbd1aaecb1f4f608d69419aa429195c', 'request_id': 'req-494b6543-df03-4284-a5ec-ccd9d3afa2fb'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:35:05.532065 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-494b6543-df03-4284-a5ec-ccd9d3afa2fb admin admin] The token cache is being invalidate because project 9cbd1aaecb1f4f608d69419aa429195c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:35:05.532389 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-494b6543-df03-4284-a5ec-ccd9d3afa2fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:35:05.533638 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-494b6543-df03-4284-a5ec-ccd9d3afa2fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:35:05.545605 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1152/2339] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:35:05 2022] DELETE /identity/v3/projects/9cbd1aaecb1f4f608d69419aa429195c => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:35:06.724949 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e14d49-ae40-4215-a979-38e396a0e7c9 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:06.725575 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e14d49-ae40-4215-a979-38e396a0e7c9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:06.725908 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e14d49-ae40-4215-a979-38e396a0e7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:06.799632 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-66e14d49-ae40-4215-a979-38e396a0e7c9 None None] MFA Rules not processed for user `1664dba0cd9b4d9da1e4b78691b8567e`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:35:06.801818 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-66e14d49-ae40-4215-a979-38e396a0e7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:35:06.984661 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1188/2340] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:35:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:35:13.369827 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b6543-df03-4284-a5ec-ccd9d3afa2fb admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:35:13.373405 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-494b6543-df03-4284-a5ec-ccd9d3afa2fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:35:13.377049 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e14d49-ae40-4215-a979-38e396a0e7c9 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:35:13.379327 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-66e14d49-ae40-4215-a979-38e396a0e7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:35:13.427296 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:35:13.510094 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b6543-df03-4284-a5ec-ccd9d3afa2fb admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:35:13.525047 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428f854-782c-4d48-8cff-93d78f98e461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:35:13.526396 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428f854-782c-4d48-8cff-93d78f98e461 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:13.526853 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428f854-782c-4d48-8cff-93d78f98e461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:13.527311 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428f854-782c-4d48-8cff-93d78f98e461 admin admin] PATH_INFO: `/v3/users/5fd88c8a7f6d49e8bb0d0045183c1beb` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:13.529177 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a428f854-782c-4d48-8cff-93d78f98e461 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fd88c8a7f6d49e8bb0d0045183c1beb)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:35:13.532202 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a428f854-782c-4d48-8cff-93d78f98e461 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:35:13.612971 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:35:13.629538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e14d49-ae40-4215-a979-38e396a0e7c9 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:35:13.637292 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c3fa81-bc7f-4413-95db-3cb2d9bf366e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6143bddc05a46ea972709576c7ad95f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:35:13.638532 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3fa81-bc7f-4413-95db-3cb2d9bf366e service placement] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:13.639238 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3fa81-bc7f-4413-95db-3cb2d9bf366e service placement] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:13.639725 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3fa81-bc7f-4413-95db-3cb2d9bf366e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:13.642611 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b3c3fa81-bc7f-4413-95db-3cb2d9bf366e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:35:13.658086 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c3fa81-bc7f-4413-95db-3cb2d9bf366e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:35:13.660558 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c3fa81-bc7f-4413-95db-3cb2d9bf366e service placement] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:35:13.674844 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a428f854-782c-4d48-8cff-93d78f98e461 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fd88c8a7f6d49e8bb0d0045183c1beb'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:35:13.676272 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a428f854-782c-4d48-8cff-93d78f98e461 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fd88c8a7f6d49e8bb0d0045183c1beb'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:35:13.688769 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1189/2341] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:35:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:35:13.716355 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a428f854-782c-4d48-8cff-93d78f98e461 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fd88c8a7f6d49e8bb0d0045183c1beb'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:35:13.727358 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a428f854-782c-4d48-8cff-93d78f98e461 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fd88c8a7f6d49e8bb0d0045183c1beb'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:35:13.752646 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a428f854-782c-4d48-8cff-93d78f98e461 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fd88c8a7f6d49e8bb0d0045183c1beb'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:35:13.783669 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1153/2342] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:35:13 2022] DELETE /identity/v3/users/5fd88c8a7f6d49e8bb0d0045183c1beb => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:35:13.786388 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66643b48-ae98-42b7-9a35-551f18601f51 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:13.787020 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66643b48-ae98-42b7-9a35-551f18601f51 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:13.787586 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66643b48-ae98-42b7-9a35-551f18601f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:13.839812 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-66643b48-ae98-42b7-9a35-551f18601f51 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:35:13.846036 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-66643b48-ae98-42b7-9a35-551f18601f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:35:13.956770 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1190/2343] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:35:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:35:13.963971 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428f854-782c-4d48-8cff-93d78f98e461 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:35:13.966761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a428f854-782c-4d48-8cff-93d78f98e461 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:35:14.199243 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428f854-782c-4d48-8cff-93d78f98e461 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:35:14.220889 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ecd42f-7396-4cce-aaf1-b98eb6d15319 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:35:14.221591 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ecd42f-7396-4cce-aaf1-b98eb6d15319 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:14.221878 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ecd42f-7396-4cce-aaf1-b98eb6d15319 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:14.222156 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ecd42f-7396-4cce-aaf1-b98eb6d15319 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:14.241977 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ecd42f-7396-4cce-aaf1-b98eb6d15319 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:35:14.244128 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ecd42f-7396-4cce-aaf1-b98eb6d15319 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:35:14.296795 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1154/2344] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:35:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:35:15.072193 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66643b48-ae98-42b7-9a35-551f18601f51 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:35:15.074747 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-66643b48-ae98-42b7-9a35-551f18601f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:35:15.092729 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66643b48-ae98-42b7-9a35-551f18601f51 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:35:15.099175 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51540ad3-cf88-439f-954e-c300ed149ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:35:15.100244 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51540ad3-cf88-439f-954e-c300ed149ed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:15.100563 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51540ad3-cf88-439f-954e-c300ed149ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:15.100854 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51540ad3-cf88-439f-954e-c300ed149ed7 admin admin] PATH_INFO: `/v3/projects/268378e6b7704c0882ba3967414051d8` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:15.102319 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51540ad3-cf88-439f-954e-c300ed149ed7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=268378e6b7704c0882ba3967414051d8)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:35:15.104676 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51540ad3-cf88-439f-954e-c300ed149ed7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:35:15.200228 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-51540ad3-cf88-439f-954e-c300ed149ed7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '268378e6b7704c0882ba3967414051d8', 'request_id': 'req-51540ad3-cf88-439f-954e-c300ed149ed7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:35:15.220840 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-51540ad3-cf88-439f-954e-c300ed149ed7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '268378e6b7704c0882ba3967414051d8', 'request_id': 'req-51540ad3-cf88-439f-954e-c300ed149ed7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:35:15.231383 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-51540ad3-cf88-439f-954e-c300ed149ed7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '268378e6b7704c0882ba3967414051d8', 'request_id': 'req-51540ad3-cf88-439f-954e-c300ed149ed7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:35:15.251202 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-51540ad3-cf88-439f-954e-c300ed149ed7 admin admin] The token cache is being invalidate because project 268378e6b7704c0882ba3967414051d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:35:15.251511 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-51540ad3-cf88-439f-954e-c300ed149ed7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:35:15.252343 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-51540ad3-cf88-439f-954e-c300ed149ed7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:35:15.254400 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1191/2345] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:35:15 2022] DELETE /identity/v3/projects/268378e6b7704c0882ba3967414051d8 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:35:15.261968 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315abe7f-065f-41c3-8c58-b5a19bec38aa None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:15.263106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315abe7f-065f-41c3-8c58-b5a19bec38aa None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:15.263691 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315abe7f-065f-41c3-8c58-b5a19bec38aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:15.301267 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-315abe7f-065f-41c3-8c58-b5a19bec38aa None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:35:15.302976 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-315abe7f-065f-41c3-8c58-b5a19bec38aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:35:15.355355 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:35:15.424287 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1155/2346] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:35:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:35:15.425155 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f91ac-2807-49fb-acc4-32ca59b94bd0 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:15.425659 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f91ac-2807-49fb-acc4-32ca59b94bd0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:15.425822 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f91ac-2807-49fb-acc4-32ca59b94bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:15.462284 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-461f91ac-2807-49fb-acc4-32ca59b94bd0 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:35:15.468364 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-461f91ac-2807-49fb-acc4-32ca59b94bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:35:15.489873 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315abe7f-065f-41c3-8c58-b5a19bec38aa None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:35:15.511497 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1192/2347] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:35:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:35:15.527185 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315abe7f-065f-41c3-8c58-b5a19bec38aa None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:35:15.537027 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97583cd-3226-4657-8382-4c9b24455caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:35:15.538733 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97583cd-3226-4657-8382-4c9b24455caf admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:15.539558 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97583cd-3226-4657-8382-4c9b24455caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:15.540311 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97583cd-3226-4657-8382-4c9b24455caf admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:15.542062 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97583cd-3226-4657-8382-4c9b24455caf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:35:15.543041 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97583cd-3226-4657-8382-4c9b24455caf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:35:15.546557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97583cd-3226-4657-8382-4c9b24455caf admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:35:15.580832 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f91ac-2807-49fb-acc4-32ca59b94bd0 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:35:15.589854 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1156/2348] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:35:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:35:15.613852 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f91ac-2807-49fb-acc4-32ca59b94bd0 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:35:15.640534 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c07fad-9b1a-44fa-868b-452a3779598f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:35:15.640534 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c07fad-9b1a-44fa-868b-452a3779598f admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:15.640534 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c07fad-9b1a-44fa-868b-452a3779598f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:15.640534 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c07fad-9b1a-44fa-868b-452a3779598f admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:15.641763 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c07fad-9b1a-44fa-868b-452a3779598f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:35:15.647808 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c07fad-9b1a-44fa-868b-452a3779598f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:35:15.710963 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1193/2349] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:35:15 2022] POST /identity/v3/projects => generated 381 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:35:15.717105 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97583cd-3226-4657-8382-4c9b24455caf admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:35:15.758927 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97583cd-3226-4657-8382-4c9b24455caf admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:35:15.928848 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bdbcee-b47a-4b1b-a910-43d041311470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:35:15.929573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bdbcee-b47a-4b1b-a910-43d041311470 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:15.929855 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bdbcee-b47a-4b1b-a910-43d041311470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:15.930128 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bdbcee-b47a-4b1b-a910-43d041311470 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:15.931043 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bdbcee-b47a-4b1b-a910-43d041311470 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:35:15.933090 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bdbcee-b47a-4b1b-a910-43d041311470 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:35:15.980860 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1157/2350] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:35:15 2022] POST /identity/v3/users => generated 291 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:35:15.986578 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c07fad-9b1a-44fa-868b-452a3779598f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:35:16.001756 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c07fad-9b1a-44fa-868b-452a3779598f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:35:16.008440 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915728ef-d506-41ba-a7f2-2bebdfb0e763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:35:16.009335 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915728ef-d506-41ba-a7f2-2bebdfb0e763 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:16.009853 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915728ef-d506-41ba-a7f2-2bebdfb0e763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:16.010340 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915728ef-d506-41ba-a7f2-2bebdfb0e763 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:16.011264 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915728ef-d506-41ba-a7f2-2bebdfb0e763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:35:16.013378 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915728ef-d506-41ba-a7f2-2bebdfb0e763 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:35:16.034613 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1194/2351] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:35:16.039450 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bdbcee-b47a-4b1b-a910-43d041311470 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:35:16.062191 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bdbcee-b47a-4b1b-a910-43d041311470 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:35:16.072710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5880c915-b0ba-4013-8d03-22348a58e89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:35:16.073957 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880c915-b0ba-4013-8d03-22348a58e89d admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:16.074570 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880c915-b0ba-4013-8d03-22348a58e89d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:16.075101 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880c915-b0ba-4013-8d03-22348a58e89d admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:16.077351 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5880c915-b0ba-4013-8d03-22348a58e89d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:35:16.081541 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5880c915-b0ba-4013-8d03-22348a58e89d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:35:16.105450 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1158/2352] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:35:16 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:35:16.110479 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915728ef-d506-41ba-a7f2-2bebdfb0e763 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:35:16.151004 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915728ef-d506-41ba-a7f2-2bebdfb0e763 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:35:16.179379 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d1c65d-1929-426e-83b7-01212e1e3d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:35:16.183360 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d1c65d-1929-426e-83b7-01212e1e3d96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:16.184375 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d1c65d-1929-426e-83b7-01212e1e3d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:16.185443 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d1c65d-1929-426e-83b7-01212e1e3d96 admin admin] PATH_INFO: `/v3/projects/120a6c484bec4e52bc5891c927647e75/users/1ad8b76baf294f91bc4c641280ab03f7/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:16.257431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d1c65d-1929-426e-83b7-01212e1e3d96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=120a6c484bec4e52bc5891c927647e75, user_id=1ad8b76baf294f91bc4c641280ab03f7, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:35:16.265685 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d1c65d-1929-426e-83b7-01212e1e3d96 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:35:16.322564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1195/2353] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:35:16 2022] PUT /identity/v3/projects/120a6c484bec4e52bc5891c927647e75/users/1ad8b76baf294f91bc4c641280ab03f7/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:35:16.330659 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d46b7-b23b-4936-9b5c-92ff7cb0952e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:16.331509 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d46b7-b23b-4936-9b5c-92ff7cb0952e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:16.331509 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d46b7-b23b-4936-9b5c-92ff7cb0952e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:16.376699 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-ad2d46b7-b23b-4936-9b5c-92ff7cb0952e None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:35:16.378044 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ad2d46b7-b23b-4936-9b5c-92ff7cb0952e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:35:16.519589 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1159/2354] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:35:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:35:16.566997 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d1c65d-1929-426e-83b7-01212e1e3d96 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:35:16.568840 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-76d1c65d-1929-426e-83b7-01212e1e3d96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:35:16.583833 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:35:16.739599 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d1c65d-1929-426e-83b7-01212e1e3d96 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:35:16.748421 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39303f-3173-43cf-b5f6-b372f9b59a1a None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:16.749661 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39303f-3173-43cf-b5f6-b372f9b59a1a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:16.751036 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39303f-3173-43cf-b5f6-b372f9b59a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:16.752877 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df5a48-2d11-4f59-9de0-0a0a8f8a4f5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:35:16.752877 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df5a48-2d11-4f59-9de0-0a0a8f8a4f5f service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:16.753271 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df5a48-2d11-4f59-9de0-0a0a8f8a4f5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:16.753549 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df5a48-2d11-4f59-9de0-0a0a8f8a4f5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:16.768576 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67df5a48-2d11-4f59-9de0-0a0a8f8a4f5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:35:16.770202 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67df5a48-2d11-4f59-9de0-0a0a8f8a4f5f service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:35:16.785299 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1196/2355] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:35:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:35:16.796199 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-3e39303f-3173-43cf-b5f6-b372f9b59a1a None None] MFA Rules not processed for user `eba91dff7cac401e81f6fb6391874af1`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:35:16.798794 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3e39303f-3173-43cf-b5f6-b372f9b59a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:35:16.941359 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:35:17.002837 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1160/2356] 10.222.0.65 () {58 vars in 1009 bytes} [Thu May 19 21:35:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:35:17.033408 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df5a48-2d11-4f59-9de0-0a0a8f8a4f5f service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:35:17.040736 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-67df5a48-2d11-4f59-9de0-0a0a8f8a4f5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:35:17.347996 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df5a48-2d11-4f59-9de0-0a0a8f8a4f5f service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:35:17.359634 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729fb732-fbbe-4a8c-a8f8-8a2a3c018af8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e07984b36934aff9052541ef2a56f36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:35:17.360339 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729fb732-fbbe-4a8c-a8f8-8a2a3c018af8 service swift] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:17.361308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729fb732-fbbe-4a8c-a8f8-8a2a3c018af8 service swift] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:17.361637 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729fb732-fbbe-4a8c-a8f8-8a2a3c018af8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:17.385157 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729fb732-fbbe-4a8c-a8f8-8a2a3c018af8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:35:17.386903 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729fb732-fbbe-4a8c-a8f8-8a2a3c018af8 service swift] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:35:17.421352 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1197/2357] 10.222.0.65 () {60 vars in 1384 bytes} [Thu May 19 21:35:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 21:35:20.240720 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a9767-928c-4e72-a101-8a926eacc190 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:20.240720 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a9767-928c-4e72-a101-8a926eacc190 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:20.240720 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a9767-928c-4e72-a101-8a926eacc190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:20.365802 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-411a9767-928c-4e72-a101-8a926eacc190 None None] MFA Rules not processed for user `1ad8b76baf294f91bc4c641280ab03f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:35:20.368385 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-411a9767-928c-4e72-a101-8a926eacc190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:35:20.596667 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1161/2358] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:35:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:35:20.626031 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729fb732-fbbe-4a8c-a8f8-8a2a3c018af8 service swift] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:35:20.628428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-729fb732-fbbe-4a8c-a8f8-8a2a3c018af8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:35:20.865962 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729fb732-fbbe-4a8c-a8f8-8a2a3c018af8 service swift] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:35:20.874499 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c817923-e215-4938-89bd-a2e03b386c54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:35:20.875508 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c817923-e215-4938-89bd-a2e03b386c54 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:20.876413 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c817923-e215-4938-89bd-a2e03b386c54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:20.877074 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c817923-e215-4938-89bd-a2e03b386c54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:20.892718 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c817923-e215-4938-89bd-a2e03b386c54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:35:20.895987 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c817923-e215-4938-89bd-a2e03b386c54 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:35:20.912923 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1198/2359] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:35:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:35:22.310490 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d5f6f-8275-4d63-9a0a-d01142192b58 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:22.312556 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d5f6f-8275-4d63-9a0a-d01142192b58 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:22.313500 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d5f6f-8275-4d63-9a0a-d01142192b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:22.350773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-852d5f6f-8275-4d63-9a0a-d01142192b58 None None] MFA Rules not processed for user `eba91dff7cac401e81f6fb6391874af1`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:35:22.353425 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-852d5f6f-8275-4d63-9a0a-d01142192b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:35:22.371305 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1162/2360] 10.222.0.65 () {58 vars in 967 bytes} [Thu May 19 21:35:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:35:22.396918 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c817923-e215-4938-89bd-a2e03b386c54 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:35:22.429774 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c817923-e215-4938-89bd-a2e03b386c54 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:35:22.449977 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd359b9e-c40b-45e2-a4ca-9a4d56016b3e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e07984b36934aff9052541ef2a56f36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:35:22.451325 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd359b9e-c40b-45e2-a4ca-9a4d56016b3e service swift] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:22.451831 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd359b9e-c40b-45e2-a4ca-9a4d56016b3e service swift] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:22.452665 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd359b9e-c40b-45e2-a4ca-9a4d56016b3e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:22.468125 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd359b9e-c40b-45e2-a4ca-9a4d56016b3e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:35:22.471091 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd359b9e-c40b-45e2-a4ca-9a4d56016b3e service swift] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:35:22.488975 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1199/2361] 10.222.0.65 () {60 vars in 1384 bytes} [Thu May 19 21:35:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 21:35:23.069912 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672da18-1ff4-437c-8e11-b9afbd09d813 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:23.070189 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672da18-1ff4-437c-8e11-b9afbd09d813 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:23.070923 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672da18-1ff4-437c-8e11-b9afbd09d813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:23.095916 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-5672da18-1ff4-437c-8e11-b9afbd09d813 None None] MFA Rules not processed for user `1ad8b76baf294f91bc4c641280ab03f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:35:23.097189 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5672da18-1ff4-437c-8e11-b9afbd09d813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:35:23.110446 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1163/2362] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:35:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:35:25.839070 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd359b9e-c40b-45e2-a4ca-9a4d56016b3e service swift] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:35:25.851336 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cd359b9e-c40b-45e2-a4ca-9a4d56016b3e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:35:25.889487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd359b9e-c40b-45e2-a4ca-9a4d56016b3e service swift] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:35:25.900042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6066bf-efe5-4d3c-82d8-64f6467db386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:35:25.900973 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6066bf-efe5-4d3c-82d8-64f6467db386 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:25.901256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6066bf-efe5-4d3c-82d8-64f6467db386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:25.901577 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6066bf-efe5-4d3c-82d8-64f6467db386 admin admin] PATH_INFO: `/v3/users/47575a76d62b4a109fda03f4bf016fc6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:25.903040 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6066bf-efe5-4d3c-82d8-64f6467db386 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47575a76d62b4a109fda03f4bf016fc6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:35:25.905304 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6066bf-efe5-4d3c-82d8-64f6467db386 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:35:25.944662 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:35:25.979417 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7f6066bf-efe5-4d3c-82d8-64f6467db386 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47575a76d62b4a109fda03f4bf016fc6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:35:26.008375 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7f6066bf-efe5-4d3c-82d8-64f6467db386 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47575a76d62b4a109fda03f4bf016fc6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:35:26.010815 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7f6066bf-efe5-4d3c-82d8-64f6467db386 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47575a76d62b4a109fda03f4bf016fc6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:35:26.011618 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7f6066bf-efe5-4d3c-82d8-64f6467db386 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47575a76d62b4a109fda03f4bf016fc6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:35:26.021851 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7f6066bf-efe5-4d3c-82d8-64f6467db386 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47575a76d62b4a109fda03f4bf016fc6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:35:26.048501 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1200/2363] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:35:25 2022] DELETE /identity/v3/users/47575a76d62b4a109fda03f4bf016fc6 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:35:26.466984 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672da18-1ff4-437c-8e11-b9afbd09d813 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:35:26.493081 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672da18-1ff4-437c-8e11-b9afbd09d813 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:35:26.620470 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db5e5f-3795-4487-8c61-3af69141285e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:35:26.621377 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db5e5f-3795-4487-8c61-3af69141285e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:26.621706 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db5e5f-3795-4487-8c61-3af69141285e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:26.622019 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db5e5f-3795-4487-8c61-3af69141285e admin admin] PATH_INFO: `/v3/projects/f9c4f4baaf8f4a258171bf97abdc8905` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:26.623788 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8db5e5f-3795-4487-8c61-3af69141285e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9c4f4baaf8f4a258171bf97abdc8905)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:35:26.625962 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8db5e5f-3795-4487-8c61-3af69141285e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:35:26.717983 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b8db5e5f-3795-4487-8c61-3af69141285e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9c4f4baaf8f4a258171bf97abdc8905', 'request_id': 'req-b8db5e5f-3795-4487-8c61-3af69141285e'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:35:26.726883 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b8db5e5f-3795-4487-8c61-3af69141285e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9c4f4baaf8f4a258171bf97abdc8905', 'request_id': 'req-b8db5e5f-3795-4487-8c61-3af69141285e'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:35:26.741730 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b8db5e5f-3795-4487-8c61-3af69141285e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9c4f4baaf8f4a258171bf97abdc8905', 'request_id': 'req-b8db5e5f-3795-4487-8c61-3af69141285e'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:35:26.767226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b8db5e5f-3795-4487-8c61-3af69141285e admin admin] The token cache is being invalidate because project f9c4f4baaf8f4a258171bf97abdc8905 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:35:26.767754 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b8db5e5f-3795-4487-8c61-3af69141285e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:35:26.769056 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b8db5e5f-3795-4487-8c61-3af69141285e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:35:26.773293 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1164/2364] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:35:26 2022] DELETE /identity/v3/projects/f9c4f4baaf8f4a258171bf97abdc8905 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:35:26.779415 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635c49c-b5d6-4846-b840-6caf93aaab48 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:26.779743 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635c49c-b5d6-4846-b840-6caf93aaab48 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:26.779974 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635c49c-b5d6-4846-b840-6caf93aaab48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:26.801699 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-2635c49c-b5d6-4846-b840-6caf93aaab48 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:35:26.803007 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2635c49c-b5d6-4846-b840-6caf93aaab48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:35:26.899904 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1201/2365] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:35:26.914258 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea1e50b-a504-4991-940b-87c1321dc60f None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:26.914733 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea1e50b-a504-4991-940b-87c1321dc60f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:26.915200 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea1e50b-a504-4991-940b-87c1321dc60f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:26.930639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:35:26.952094 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-cea1e50b-a504-4991-940b-87c1321dc60f None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:35:26.953722 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-cea1e50b-a504-4991-940b-87c1321dc60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:35:26.962375 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1165/2366] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:35:26.967552 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635c49c-b5d6-4846-b840-6caf93aaab48 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:35:26.978944 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635c49c-b5d6-4846-b840-6caf93aaab48 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:35:26.984185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589240a-e246-48a1-98fc-910692c6d995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:35:26.984888 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589240a-e246-48a1-98fc-910692c6d995 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:26.985177 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589240a-e246-48a1-98fc-910692c6d995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:26.985458 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589240a-e246-48a1-98fc-910692c6d995 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:26.986267 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5589240a-e246-48a1-98fc-910692c6d995 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:35:26.986588 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5589240a-e246-48a1-98fc-910692c6d995 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:35:26.988262 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5589240a-e246-48a1-98fc-910692c6d995 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:35:27.006426 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1202/2367] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:35:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:35:27.011657 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea1e50b-a504-4991-940b-87c1321dc60f None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:35:27.038833 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea1e50b-a504-4991-940b-87c1321dc60f None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:35:27.047404 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a9c53-5017-48a0-9f57-42d849a754e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:35:27.048832 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a9c53-5017-48a0-9f57-42d849a754e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:27.049357 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a9c53-5017-48a0-9f57-42d849a754e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:27.049826 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a9c53-5017-48a0-9f57-42d849a754e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:27.051062 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22a9c53-5017-48a0-9f57-42d849a754e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:35:27.054652 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22a9c53-5017-48a0-9f57-42d849a754e1 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:35:27.113521 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1166/2368] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:35:27 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:35:27.119593 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589240a-e246-48a1-98fc-910692c6d995 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:35:27.135373 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589240a-e246-48a1-98fc-910692c6d995 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:35:27.249674 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6dd510-6a6a-4444-a1a5-5cf6a24dffd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:35:27.250830 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6dd510-6a6a-4444-a1a5-5cf6a24dffd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:27.251319 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6dd510-6a6a-4444-a1a5-5cf6a24dffd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:27.251815 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6dd510-6a6a-4444-a1a5-5cf6a24dffd4 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:27.253104 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6dd510-6a6a-4444-a1a5-5cf6a24dffd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:35:27.256798 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6dd510-6a6a-4444-a1a5-5cf6a24dffd4 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:35:27.287942 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1203/2369] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:35:27 2022] POST /identity/v3/users => generated 287 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:35:27.293598 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a9c53-5017-48a0-9f57-42d849a754e1 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:35:27.317938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a9c53-5017-48a0-9f57-42d849a754e1 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:35:27.327053 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c29eeff-0825-4afe-bb7f-77fc0ddfb8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:35:27.328058 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29eeff-0825-4afe-bb7f-77fc0ddfb8e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:27.330335 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29eeff-0825-4afe-bb7f-77fc0ddfb8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:27.330335 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29eeff-0825-4afe-bb7f-77fc0ddfb8e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:27.330335 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c29eeff-0825-4afe-bb7f-77fc0ddfb8e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:35:27.332956 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c29eeff-0825-4afe-bb7f-77fc0ddfb8e0 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:35:27.370925 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1167/2370] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:35:27.376636 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6dd510-6a6a-4444-a1a5-5cf6a24dffd4 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:35:27.410615 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6dd510-6a6a-4444-a1a5-5cf6a24dffd4 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:35:27.415718 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff8b0c8-311f-4021-b695-2144828db5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:35:27.416634 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8b0c8-311f-4021-b695-2144828db5e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:27.417174 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8b0c8-311f-4021-b695-2144828db5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:27.417606 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8b0c8-311f-4021-b695-2144828db5e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:27.418539 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff8b0c8-311f-4021-b695-2144828db5e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:35:27.420408 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff8b0c8-311f-4021-b695-2144828db5e4 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:35:27.433304 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1204/2371] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:35:27.439780 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c29eeff-0825-4afe-bb7f-77fc0ddfb8e0 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:35:27.452705 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c29eeff-0825-4afe-bb7f-77fc0ddfb8e0 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:35:27.458042 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279cd0f8-7ad2-4fca-bbe1-61f9be48cbb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:35:27.459017 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279cd0f8-7ad2-4fca-bbe1-61f9be48cbb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:27.459303 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279cd0f8-7ad2-4fca-bbe1-61f9be48cbb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:27.459583 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279cd0f8-7ad2-4fca-bbe1-61f9be48cbb7 admin admin] PATH_INFO: `/v3/projects/c887b19b903a4702b2812ef9b5b88ddc/users/4113ddb776d144f0ae2bb2b7d28e775a/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:27.495455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279cd0f8-7ad2-4fca-bbe1-61f9be48cbb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c887b19b903a4702b2812ef9b5b88ddc, user_id=4113ddb776d144f0ae2bb2b7d28e775a, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:35:27.502968 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279cd0f8-7ad2-4fca-bbe1-61f9be48cbb7 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:35:27.537294 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1168/2372] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:35:27 2022] PUT /identity/v3/projects/c887b19b903a4702b2812ef9b5b88ddc/users/4113ddb776d144f0ae2bb2b7d28e775a/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:35:27.562529 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbe59e-1030-4d50-8c67-5b687f2c0bcf None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:27.563216 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbe59e-1030-4d50-8c67-5b687f2c0bcf None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:27.563958 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbe59e-1030-4d50-8c67-5b687f2c0bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:27.594760 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:35:27.621997 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-8afbe59e-1030-4d50-8c67-5b687f2c0bcf None None] MFA Rules not processed for user `4113ddb776d144f0ae2bb2b7d28e775a`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:35:27.623407 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8afbe59e-1030-4d50-8c67-5b687f2c0bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:35:27.729901 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1205/2373] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:35:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:35:27.744568 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279cd0f8-7ad2-4fca-bbe1-61f9be48cbb7 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:35:27.749915 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-279cd0f8-7ad2-4fca-bbe1-61f9be48cbb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:35:27.898410 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279cd0f8-7ad2-4fca-bbe1-61f9be48cbb7 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:35:27.906206 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b5abb-1344-4224-bf29-cc30f5c3726c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:35:27.907798 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b5abb-1344-4224-bf29-cc30f5c3726c service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:27.908435 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b5abb-1344-4224-bf29-cc30f5c3726c service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:27.909079 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b5abb-1344-4224-bf29-cc30f5c3726c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:27.921429 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6b5abb-1344-4224-bf29-cc30f5c3726c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:35:27.922969 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6b5abb-1344-4224-bf29-cc30f5c3726c service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:35:27.934772 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1169/2374] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:35:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:35:30.625258 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d8a1f-2df7-47ab-a6c9-71f6f8d999e9 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:30.625860 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d8a1f-2df7-47ab-a6c9-71f6f8d999e9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:30.625860 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d8a1f-2df7-47ab-a6c9-71f6f8d999e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:30.647817 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e26d8a1f-2df7-47ab-a6c9-71f6f8d999e9 None None] MFA Rules not processed for user `4113ddb776d144f0ae2bb2b7d28e775a`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:35:30.649420 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e26d8a1f-2df7-47ab-a6c9-71f6f8d999e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:35:30.657510 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1206/2375] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:35:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:35:37.930028 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=52,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:35:40.640627 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=32 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:35:41.846370 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b5abb-1344-4224-bf29-cc30f5c3726c service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:35:41.848203 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ba6b5abb-1344-4224-bf29-cc30f5c3726c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:35:41.944253 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b5abb-1344-4224-bf29-cc30f5c3726c service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:35:41.948697 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8c569d-38b5-4ae1-9ceb-4ad36c5fd3d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:35:41.949284 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c569d-38b5-4ae1-9ceb-4ad36c5fd3d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:35:41.949597 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c569d-38b5-4ae1-9ceb-4ad36c5fd3d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:35:41.950044 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c569d-38b5-4ae1-9ceb-4ad36c5fd3d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:35:41.952500 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-fa8c569d-38b5-4ae1-9ceb-4ad36c5fd3d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:35:42.042656 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8c569d-38b5-4ae1-9ceb-4ad36c5fd3d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:35:42.044233 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8c569d-38b5-4ae1-9ceb-4ad36c5fd3d9 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:35:42.061925 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1170/2376] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:35:41 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:35:52.054078 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=50 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:36:04.907245 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d8a1f-2df7-47ab-a6c9-71f6f8d999e9 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:36:04.919724 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d8a1f-2df7-47ab-a6c9-71f6f8d999e9 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:36:04.925368 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf0fce9-34f5-4cbd-adea-35ef8428d7f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:36:04.926595 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0fce9-34f5-4cbd-adea-35ef8428d7f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:36:04.927118 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0fce9-34f5-4cbd-adea-35ef8428d7f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:36:04.927612 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0fce9-34f5-4cbd-adea-35ef8428d7f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:36:04.931077 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-abf0fce9-34f5-4cbd-adea-35ef8428d7f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:36:05.041073 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf0fce9-34f5-4cbd-adea-35ef8428d7f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:36:05.043952 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf0fce9-34f5-4cbd-adea-35ef8428d7f1 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:36:05.064809 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1207/2377] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:36:04 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:36:06.637807 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237ec84-51dd-4227-9320-f82551cb65a7 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:36:06.638783 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237ec84-51dd-4227-9320-f82551cb65a7 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:36:06.638783 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237ec84-51dd-4227-9320-f82551cb65a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:36:06.670803 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-4237ec84-51dd-4227-9320-f82551cb65a7 None None] MFA Rules not processed for user `eba91dff7cac401e81f6fb6391874af1`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:36:06.674720 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4237ec84-51dd-4227-9320-f82551cb65a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:36:06.771254 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1171/2378] 10.222.0.65 () {58 vars in 1009 bytes} [Thu May 19 21:36:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:36:06.819835 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf0fce9-34f5-4cbd-adea-35ef8428d7f1 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:36:06.822556 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-abf0fce9-34f5-4cbd-adea-35ef8428d7f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:36:06.923670 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf0fce9-34f5-4cbd-adea-35ef8428d7f1 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:36:06.932618 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0fa45-7eae-484b-95c8-de0e0e33d9c7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e07984b36934aff9052541ef2a56f36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:36:06.933807 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0fa45-7eae-484b-95c8-de0e0e33d9c7 service swift] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:36:06.935241 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0fa45-7eae-484b-95c8-de0e0e33d9c7 service swift] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:36:06.935723 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0fa45-7eae-484b-95c8-de0e0e33d9c7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:36:06.951489 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0fa45-7eae-484b-95c8-de0e0e33d9c7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:36:06.954419 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0fa45-7eae-484b-95c8-de0e0e33d9c7 service swift] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:36:06.971113 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1208/2379] 10.222.0.65 () {60 vars in 1384 bytes} [Thu May 19 21:36:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 19 21:36:07.996295 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c410fad7-ca82-4299-94b3-0aa57d19273b None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:36:07.996968 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c410fad7-ca82-4299-94b3-0aa57d19273b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:36:07.996968 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c410fad7-ca82-4299-94b3-0aa57d19273b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:36:08.019829 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-c410fad7-ca82-4299-94b3-0aa57d19273b None None] MFA Rules not processed for user `eba91dff7cac401e81f6fb6391874af1`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:36:08.021176 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c410fad7-ca82-4299-94b3-0aa57d19273b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:36:08.033639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1172/2380] 10.222.0.65 () {58 vars in 967 bytes} [Thu May 19 21:36:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:36:08.050955 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0fa45-7eae-484b-95c8-de0e0e33d9c7 service swift] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:36:08.065583 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0fa45-7eae-484b-95c8-de0e0e33d9c7 service swift] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:36:08.072873 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679eba99-cd01-47d8-a544-37497084ffb0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e07984b36934aff9052541ef2a56f36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:36:08.074174 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679eba99-cd01-47d8-a544-37497084ffb0 service swift] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:36:08.076735 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679eba99-cd01-47d8-a544-37497084ffb0 service swift] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:36:08.077420 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679eba99-cd01-47d8-a544-37497084ffb0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:36:08.091071 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679eba99-cd01-47d8-a544-37497084ffb0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:36:08.093396 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679eba99-cd01-47d8-a544-37497084ffb0 service swift] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:36:08.116786 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1209/2381] 10.222.0.65 () {60 vars in 1384 bytes} [Thu May 19 21:36:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 19 21:36:08.439264 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e01bb-bbba-4276-8ee4-77885d5da4a6 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:36:08.439716 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e01bb-bbba-4276-8ee4-77885d5da4a6 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:36:08.440440 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e01bb-bbba-4276-8ee4-77885d5da4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:36:08.480827 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-5b5e01bb-bbba-4276-8ee4-77885d5da4a6 None None] MFA Rules not processed for user `eba91dff7cac401e81f6fb6391874af1`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:36:08.484180 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5b5e01bb-bbba-4276-8ee4-77885d5da4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:36:08.498776 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1173/2382] 10.222.0.65 () {58 vars in 1009 bytes} [Thu May 19 21:36:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:36:08.522573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679eba99-cd01-47d8-a544-37497084ffb0 service swift] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:36:08.541846 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679eba99-cd01-47d8-a544-37497084ffb0 service swift] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:36:08.552371 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7302c1-ac64-4e5a-89b9-04eb736fc507 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e07984b36934aff9052541ef2a56f36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:36:08.553646 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7302c1-ac64-4e5a-89b9-04eb736fc507 service swift] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:36:08.554180 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7302c1-ac64-4e5a-89b9-04eb736fc507 service swift] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:36:08.554742 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7302c1-ac64-4e5a-89b9-04eb736fc507 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:36:08.571657 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7302c1-ac64-4e5a-89b9-04eb736fc507 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:36:08.574031 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7302c1-ac64-4e5a-89b9-04eb736fc507 service swift] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:36:08.594929 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1210/2383] 10.222.0.65 () {60 vars in 1384 bytes} [Thu May 19 21:36:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 21:36:09.221262 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72fa5e-d136-4a38-a680-d5cc3bb0775f None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:36:09.221811 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72fa5e-d136-4a38-a680-d5cc3bb0775f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:36:09.221811 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72fa5e-d136-4a38-a680-d5cc3bb0775f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:36:09.242067 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-cd72fa5e-d136-4a38-a680-d5cc3bb0775f None None] MFA Rules not processed for user `eba91dff7cac401e81f6fb6391874af1`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:36:09.243427 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-cd72fa5e-d136-4a38-a680-d5cc3bb0775f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:36:09.252473 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1174/2384] 10.222.0.65 () {58 vars in 967 bytes} [Thu May 19 21:36:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:36:09.266444 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7302c1-ac64-4e5a-89b9-04eb736fc507 service swift] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:36:09.277766 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7302c1-ac64-4e5a-89b9-04eb736fc507 service swift] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:36:09.283702 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feed882b-57d9-4d74-95d9-872196728959 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e07984b36934aff9052541ef2a56f36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:36:09.284304 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed882b-57d9-4d74-95d9-872196728959 service swift] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:36:09.284609 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed882b-57d9-4d74-95d9-872196728959 service swift] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:36:09.285019 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed882b-57d9-4d74-95d9-872196728959 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:36:09.301189 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feed882b-57d9-4d74-95d9-872196728959 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:36:09.303287 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feed882b-57d9-4d74-95d9-872196728959 service swift] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:36:09.314922 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1211/2385] 10.222.0.65 () {60 vars in 1384 bytes} [Thu May 19 21:36:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 21:36:12.035616 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd72fa5e-d136-4a38-a680-d5cc3bb0775f None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:36:12.144673 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd72fa5e-d136-4a38-a680-d5cc3bb0775f None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:36:12.149945 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d03a6-76ce-4f69-ba48-25e9f684bd67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:36:12.150878 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d03a6-76ce-4f69-ba48-25e9f684bd67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:36:12.151161 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d03a6-76ce-4f69-ba48-25e9f684bd67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:36:12.151692 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d03a6-76ce-4f69-ba48-25e9f684bd67 admin admin] PATH_INFO: `/v3/users/4113ddb776d144f0ae2bb2b7d28e775a` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:36:12.153786 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0d03a6-76ce-4f69-ba48-25e9f684bd67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4113ddb776d144f0ae2bb2b7d28e775a)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:36:12.157098 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0d03a6-76ce-4f69-ba48-25e9f684bd67 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:36:12.221900 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0c0d03a6-76ce-4f69-ba48-25e9f684bd67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4113ddb776d144f0ae2bb2b7d28e775a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:36:12.222729 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0c0d03a6-76ce-4f69-ba48-25e9f684bd67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4113ddb776d144f0ae2bb2b7d28e775a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:36:12.237963 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0c0d03a6-76ce-4f69-ba48-25e9f684bd67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4113ddb776d144f0ae2bb2b7d28e775a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:36:12.247452 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0c0d03a6-76ce-4f69-ba48-25e9f684bd67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4113ddb776d144f0ae2bb2b7d28e775a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:36:12.260785 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0c0d03a6-76ce-4f69-ba48-25e9f684bd67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4113ddb776d144f0ae2bb2b7d28e775a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:36:12.271784 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1175/2386] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:36:12 2022] DELETE /identity/v3/users/4113ddb776d144f0ae2bb2b7d28e775a => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:36:12.278146 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22913e41-f778-4920-8f0f-7bdd33f62cef None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:36:12.278948 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22913e41-f778-4920-8f0f-7bdd33f62cef None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:36:12.278948 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22913e41-f778-4920-8f0f-7bdd33f62cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:36:12.311061 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-22913e41-f778-4920-8f0f-7bdd33f62cef None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:36:12.313999 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-22913e41-f778-4920-8f0f-7bdd33f62cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:36:12.371841 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:36:12.403009 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1212/2387] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:36:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:36:12.414538 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d03a6-76ce-4f69-ba48-25e9f684bd67 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:36:12.416067 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0c0d03a6-76ce-4f69-ba48-25e9f684bd67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:36:12.423023 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:36:12.497420 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d03a6-76ce-4f69-ba48-25e9f684bd67 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:36:12.501292 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6a3a7-8c91-4fdd-ae3c-db7ef3aa009d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:36:12.501956 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6a3a7-8c91-4fdd-ae3c-db7ef3aa009d service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:36:12.502259 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6a3a7-8c91-4fdd-ae3c-db7ef3aa009d service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:36:12.502547 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6a3a7-8c91-4fdd-ae3c-db7ef3aa009d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:36:12.513362 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f6a3a7-8c91-4fdd-ae3c-db7ef3aa009d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:36:12.514889 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f6a3a7-8c91-4fdd-ae3c-db7ef3aa009d service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:36:12.525670 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1176/2388] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:36:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:36:12.730694 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22913e41-f778-4920-8f0f-7bdd33f62cef None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:36:12.732418 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-22913e41-f778-4920-8f0f-7bdd33f62cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:36:12.761647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22913e41-f778-4920-8f0f-7bdd33f62cef None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:36:12.767927 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a84ece7-865a-48ca-ba23-61afa0323fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:36:12.768658 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84ece7-865a-48ca-ba23-61afa0323fef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:36:12.768994 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84ece7-865a-48ca-ba23-61afa0323fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:36:12.769409 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84ece7-865a-48ca-ba23-61afa0323fef admin admin] PATH_INFO: `/v3/projects/c887b19b903a4702b2812ef9b5b88ddc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:36:12.771221 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a84ece7-865a-48ca-ba23-61afa0323fef admin admin] RBAC: Authorizing `identity:delete_project(project_id=c887b19b903a4702b2812ef9b5b88ddc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:36:12.773483 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a84ece7-865a-48ca-ba23-61afa0323fef admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:36:12.841849 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8a84ece7-865a-48ca-ba23-61afa0323fef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c887b19b903a4702b2812ef9b5b88ddc', 'request_id': 'req-8a84ece7-865a-48ca-ba23-61afa0323fef'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:36:12.864707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8a84ece7-865a-48ca-ba23-61afa0323fef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c887b19b903a4702b2812ef9b5b88ddc', 'request_id': 'req-8a84ece7-865a-48ca-ba23-61afa0323fef'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:36:12.876362 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8a84ece7-865a-48ca-ba23-61afa0323fef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c887b19b903a4702b2812ef9b5b88ddc', 'request_id': 'req-8a84ece7-865a-48ca-ba23-61afa0323fef'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:36:12.897713 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8a84ece7-865a-48ca-ba23-61afa0323fef admin admin] The token cache is being invalidate because project c887b19b903a4702b2812ef9b5b88ddc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:36:12.898341 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8a84ece7-865a-48ca-ba23-61afa0323fef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:36:12.899671 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8a84ece7-865a-48ca-ba23-61afa0323fef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:36:12.902623 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1213/2389] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:36:12 2022] DELETE /identity/v3/projects/c887b19b903a4702b2812ef9b5b88ddc => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:36:22.521042 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=24 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:36:22.893713 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=34,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:36:36.669983 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11e008-b4fe-4841-bfee-e780b494f572 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:36:36.670673 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11e008-b4fe-4841-bfee-e780b494f572 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:36:36.671120 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11e008-b4fe-4841-bfee-e780b494f572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:36:36.694640 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-2a11e008-b4fe-4841-bfee-e780b494f572 None None] MFA Rules not processed for user `eba91dff7cac401e81f6fb6391874af1`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:36:36.696262 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-2a11e008-b4fe-4841-bfee-e780b494f572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:36:36.765511 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1177/2390] 10.222.0.65 () {58 vars in 1009 bytes} [Thu May 19 21:36:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:36:36.778187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a84ece7-865a-48ca-ba23-61afa0323fef admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:36:36.780236 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8a84ece7-865a-48ca-ba23-61afa0323fef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:36:36.876266 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a84ece7-865a-48ca-ba23-61afa0323fef admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:36:36.882588 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa742c29-b3c7-4c5d-85e3-be316454aaa6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e07984b36934aff9052541ef2a56f36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:36:36.883260 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa742c29-b3c7-4c5d-85e3-be316454aaa6 service swift] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:36:36.883543 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa742c29-b3c7-4c5d-85e3-be316454aaa6 service swift] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:36:36.884270 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa742c29-b3c7-4c5d-85e3-be316454aaa6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:36:36.902406 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa742c29-b3c7-4c5d-85e3-be316454aaa6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:36:36.904336 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa742c29-b3c7-4c5d-85e3-be316454aaa6 service swift] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:36:36.915779 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1214/2391] 10.222.0.65 () {60 vars in 1384 bytes} [Thu May 19 21:36:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 21:36:46.755410 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=25 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:36:46.911333 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=32 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:36:59.634817 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344319f-b6eb-42d1-8008-534b293c66cf None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:36:59.636044 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344319f-b6eb-42d1-8008-534b293c66cf None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:36:59.636044 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344319f-b6eb-42d1-8008-534b293c66cf None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:36:59.637084 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1178/2392] 10.222.0.65 () {54 vars in 921 bytes} [Thu May 19 21:36:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:36:59.644422 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0050f-749b-469a-953a-aef0c789ae40 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:36:59.644997 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0050f-749b-469a-953a-aef0c789ae40 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:36:59.644997 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0050f-749b-469a-953a-aef0c789ae40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:36:59.711667 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-34b0050f-749b-469a-953a-aef0c789ae40 None None] MFA Rules not processed for user `e254671b21fe4661a381b9521810cf80`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:36:59.713004 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-34b0050f-749b-469a-953a-aef0c789ae40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:36:59.793434 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1215/2393] 10.222.0.65 () {58 vars in 1018 bytes} [Thu May 19 21:36:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:36:59.805268 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344319f-b6eb-42d1-8008-534b293c66cf None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:36:59.807290 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7344319f-b6eb-42d1-8008-534b293c66cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:36:59.877160 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344319f-b6eb-42d1-8008-534b293c66cf None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:36:59.881050 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1f9880-43d9-42ec-b065-504300f88167 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea874ca4088d4e6c83c9d0497323cf6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:36:59.881761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f9880-43d9-42ec-b065-504300f88167 service glance] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:36:59.882043 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f9880-43d9-42ec-b065-504300f88167 service glance] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:36:59.882403 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f9880-43d9-42ec-b065-504300f88167 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:36:59.894958 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1f9880-43d9-42ec-b065-504300f88167 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:36:59.896405 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1f9880-43d9-42ec-b065-504300f88167 service glance] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:36:59.908784 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1179/2394] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:36:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:36:59.985779 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db7b60-1271-46b6-9648-9811683adbc4 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:36:59.986081 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db7b60-1271-46b6-9648-9811683adbc4 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:36:59.986405 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db7b60-1271-46b6-9648-9811683adbc4 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:36:59.987812 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1216/2395] 10.222.0.65 () {54 vars in 921 bytes} [Thu May 19 21:36:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:36:59.993418 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf134d-94eb-43c6-a08e-28dda9a44887 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:36:59.993712 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf134d-94eb-43c6-a08e-28dda9a44887 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:36:59.993993 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf134d-94eb-43c6-a08e-28dda9a44887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:37:00.020054 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-fdbf134d-94eb-43c6-a08e-28dda9a44887 None None] MFA Rules not processed for user `2201e66caa4b4a57be09d845d7cb0023`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:37:00.021730 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-fdbf134d-94eb-43c6-a08e-28dda9a44887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:37:00.082720 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1180/2396] 10.222.0.65 () {58 vars in 1018 bytes} [Thu May 19 21:36:59 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:37:00.095597 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db7b60-1271-46b6-9648-9811683adbc4 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:37:00.098374 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-67db7b60-1271-46b6-9648-9811683adbc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:37:00.114886 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db7b60-1271-46b6-9648-9811683adbc4 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:37:00.118932 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d08fb-7ef3-440f-a351-d9df3d1600e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:37:00.119661 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d08fb-7ef3-440f-a351-d9df3d1600e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:37:00.119941 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d08fb-7ef3-440f-a351-d9df3d1600e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:37:00.120215 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d08fb-7ef3-440f-a351-d9df3d1600e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:37:00.128145 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816d08fb-7ef3-440f-a351-d9df3d1600e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:37:00.129635 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816d08fb-7ef3-440f-a351-d9df3d1600e8 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:37:00.139433 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1217/2397] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:37:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:37:00.237181 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0b95b-870b-490d-8f4e-6f54d49995a1 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:37:00.237568 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0b95b-870b-490d-8f4e-6f54d49995a1 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:37:00.237867 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0b95b-870b-490d-8f4e-6f54d49995a1 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:37:00.239358 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1181/2398] 10.222.0.65 () {54 vars in 921 bytes} [Thu May 19 21:37:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:37:00.245346 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3283a80-8278-4a52-95d1-6fee16baa0dd None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:37:00.245633 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3283a80-8278-4a52-95d1-6fee16baa0dd None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:37:00.245913 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3283a80-8278-4a52-95d1-6fee16baa0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:37:00.275237 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c3283a80-8278-4a52-95d1-6fee16baa0dd None None] MFA Rules not processed for user `e254671b21fe4661a381b9521810cf80`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:37:00.276805 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c3283a80-8278-4a52-95d1-6fee16baa0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:37:00.286925 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1218/2399] 10.222.0.65 () {58 vars in 1018 bytes} [Thu May 19 21:37:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:37:00.300945 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b0b95b-870b-490d-8f4e-6f54d49995a1 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:37:00.315697 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b0b95b-870b-490d-8f4e-6f54d49995a1 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:37:00.321667 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff05afc9-2420-4158-869e-23376e2471b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:37:00.322607 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05afc9-2420-4158-869e-23376e2471b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:37:00.323026 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05afc9-2420-4158-869e-23376e2471b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:37:00.323411 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05afc9-2420-4158-869e-23376e2471b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:37:00.334459 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff05afc9-2420-4158-869e-23376e2471b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:37:00.335901 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff05afc9-2420-4158-869e-23376e2471b8 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:37:00.347778 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1182/2400] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:37:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:37:04.529966 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d9c56-43d5-4620-8d90-df8cd8ba5d84 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:37:04.530477 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d9c56-43d5-4620-8d90-df8cd8ba5d84 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:37:04.530588 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d9c56-43d5-4620-8d90-df8cd8ba5d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:37:04.552027 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-8f7d9c56-43d5-4620-8d90-df8cd8ba5d84 None None] MFA Rules not processed for user `eba91dff7cac401e81f6fb6391874af1`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:37:04.553565 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8f7d9c56-43d5-4620-8d90-df8cd8ba5d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:37:04.561974 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1219/2401] 10.222.0.65 () {58 vars in 1009 bytes} [Thu May 19 21:37:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:37:04.575353 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff05afc9-2420-4158-869e-23376e2471b8 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:37:04.589026 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff05afc9-2420-4158-869e-23376e2471b8 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:37:04.598119 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab5759-613a-44bc-8efd-71a10c4228c2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e07984b36934aff9052541ef2a56f36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:37:04.599209 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab5759-613a-44bc-8efd-71a10c4228c2 service swift] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:37:04.599901 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab5759-613a-44bc-8efd-71a10c4228c2 service swift] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:37:04.600651 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab5759-613a-44bc-8efd-71a10c4228c2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:37:04.612623 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dab5759-613a-44bc-8efd-71a10c4228c2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:37:04.615428 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dab5759-613a-44bc-8efd-71a10c4228c2 service swift] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:37:04.650333 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1183/2402] 10.222.0.65 () {60 vars in 1384 bytes} [Thu May 19 21:37:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 21:37:10.471108 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d9c56-43d5-4620-8d90-df8cd8ba5d84 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:37:10.484285 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d9c56-43d5-4620-8d90-df8cd8ba5d84 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:37:10.489361 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b956d-0da2-4f0e-945c-c47fd01271ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:37:10.490022 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b956d-0da2-4f0e-945c-c47fd01271ca service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:37:10.490336 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b956d-0da2-4f0e-945c-c47fd01271ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:37:10.490584 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b956d-0da2-4f0e-945c-c47fd01271ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:37:10.492762 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e69b956d-0da2-4f0e-945c-c47fd01271ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:37:10.511357 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69b956d-0da2-4f0e-945c-c47fd01271ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:37:10.512838 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69b956d-0da2-4f0e-945c-c47fd01271ca service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:37:10.524553 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1220/2403] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:37:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:37:11.712308 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab5759-613a-44bc-8efd-71a10c4228c2 service swift] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:37:11.714282 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9dab5759-613a-44bc-8efd-71a10c4228c2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:37:11.728133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab5759-613a-44bc-8efd-71a10c4228c2 service swift] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:37:11.732494 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b75eadc-ce0b-4883-be90-1e3abcffe128 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:37:11.733152 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75eadc-ce0b-4883-be90-1e3abcffe128 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:37:11.733452 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75eadc-ce0b-4883-be90-1e3abcffe128 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:37:11.733782 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75eadc-ce0b-4883-be90-1e3abcffe128 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:37:11.736385 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3b75eadc-ce0b-4883-be90-1e3abcffe128 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:37:11.754197 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b75eadc-ce0b-4883-be90-1e3abcffe128 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:37:11.755694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b75eadc-ce0b-4883-be90-1e3abcffe128 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:37:11.771712 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1184/2404] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:37:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:37:20.519562 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=64 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:37:20.560309 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1203f3-2299-43b0-817f-68bfe8105aa2 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:37:20.560562 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1203f3-2299-43b0-817f-68bfe8105aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:37:20.560871 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1203f3-2299-43b0-817f-68bfe8105aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:37:20.580489 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-1d1203f3-2299-43b0-817f-68bfe8105aa2 None None] MFA Rules not processed for user `1ad8b76baf294f91bc4c641280ab03f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:37:20.581733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1d1203f3-2299-43b0-817f-68bfe8105aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:37:20.644654 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1221/2405] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:37:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:37:20.923569 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b75eadc-ce0b-4883-be90-1e3abcffe128 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:37:20.933360 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b75eadc-ce0b-4883-be90-1e3abcffe128 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:37:20.937066 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf71b0-a5e7-4039-a482-34bb1a785f74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:37:20.937713 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf71b0-a5e7-4039-a482-34bb1a785f74 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:37:20.937996 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf71b0-a5e7-4039-a482-34bb1a785f74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:37:20.938282 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf71b0-a5e7-4039-a482-34bb1a785f74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:37:20.940123 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ecdf71b0-a5e7-4039-a482-34bb1a785f74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:37:20.953896 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf71b0-a5e7-4039-a482-34bb1a785f74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:37:20.956164 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf71b0-a5e7-4039-a482-34bb1a785f74 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:37:20.971747 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1185/2406] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:37:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:37:21.295427 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1203f3-2299-43b0-817f-68bfe8105aa2 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:37:21.296801 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1d1203f3-2299-43b0-817f-68bfe8105aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:37:21.372743 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1203f3-2299-43b0-817f-68bfe8105aa2 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:37:21.377993 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5607d9f4-a8b5-459d-89df-2bb0c3b34bbd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6143bddc05a46ea972709576c7ad95f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:37:21.378720 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5607d9f4-a8b5-459d-89df-2bb0c3b34bbd service placement] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:37:21.379005 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5607d9f4-a8b5-459d-89df-2bb0c3b34bbd service placement] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:37:21.379280 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5607d9f4-a8b5-459d-89df-2bb0c3b34bbd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:37:21.381179 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5607d9f4-a8b5-459d-89df-2bb0c3b34bbd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:37:21.393208 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5607d9f4-a8b5-459d-89df-2bb0c3b34bbd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:37:21.394846 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5607d9f4-a8b5-459d-89df-2bb0c3b34bbd service placement] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:37:21.405262 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1222/2407] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:37:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:37:27.291458 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf71b0-a5e7-4039-a482-34bb1a785f74 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:37:27.293462 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ecdf71b0-a5e7-4039-a482-34bb1a785f74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:37:27.304896 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=74 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:37:27.314782 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf71b0-a5e7-4039-a482-34bb1a785f74 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:37:27.322490 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee99f97-7b42-4702-bfc0-8e145c7035b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:37:27.323531 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee99f97-7b42-4702-bfc0-8e145c7035b0 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:37:27.324018 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee99f97-7b42-4702-bfc0-8e145c7035b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:37:27.324483 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee99f97-7b42-4702-bfc0-8e145c7035b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:37:27.327238 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-fee99f97-7b42-4702-bfc0-8e145c7035b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:37:27.405196 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee99f97-7b42-4702-bfc0-8e145c7035b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:37:27.406902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee99f97-7b42-4702-bfc0-8e145c7035b0 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:37:27.417391 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1186/2408] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:37:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:37:28.527479 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5607d9f4-a8b5-459d-89df-2bb0c3b34bbd service placement] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:37:28.539279 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5607d9f4-a8b5-459d-89df-2bb0c3b34bbd service placement] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:37:28.545174 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9906fa9b-2ff9-4254-8029-2e860be9f9ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:37:28.545869 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9906fa9b-2ff9-4254-8029-2e860be9f9ac service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:37:28.546198 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9906fa9b-2ff9-4254-8029-2e860be9f9ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:37:28.546489 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9906fa9b-2ff9-4254-8029-2e860be9f9ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:37:28.548587 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9906fa9b-2ff9-4254-8029-2e860be9f9ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:37:28.560127 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9906fa9b-2ff9-4254-8029-2e860be9f9ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:37:28.561642 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9906fa9b-2ff9-4254-8029-2e860be9f9ac service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:37:28.571856 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1223/2409] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:37:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:37:37.412557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=27 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:37:38.567720 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=63 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:37:38.724450 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d08bbe-0c5d-45b2-bfac-895b9664478d None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:37:38.724965 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d08bbe-0c5d-45b2-bfac-895b9664478d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:37:38.724965 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d08bbe-0c5d-45b2-bfac-895b9664478d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:37:38.744999 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-f3d08bbe-0c5d-45b2-bfac-895b9664478d None None] MFA Rules not processed for user `eba91dff7cac401e81f6fb6391874af1`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:37:38.746221 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f3d08bbe-0c5d-45b2-bfac-895b9664478d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:37:38.754806 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1187/2410] 10.222.0.65 () {58 vars in 967 bytes} [Thu May 19 21:37:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:37:38.776663 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9906fa9b-2ff9-4254-8029-2e860be9f9ac service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:37:38.792178 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9906fa9b-2ff9-4254-8029-2e860be9f9ac service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:37:38.797525 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a5c73-d92a-4b70-9449-2c2de9c27d3c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e07984b36934aff9052541ef2a56f36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:37:38.798177 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a5c73-d92a-4b70-9449-2c2de9c27d3c service swift] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:37:38.798496 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a5c73-d92a-4b70-9449-2c2de9c27d3c service swift] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:37:38.798774 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a5c73-d92a-4b70-9449-2c2de9c27d3c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:37:38.807893 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a5c73-d92a-4b70-9449-2c2de9c27d3c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:37:38.809520 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a5c73-d92a-4b70-9449-2c2de9c27d3c service swift] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:37:38.817939 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1224/2411] 10.222.0.65 () {60 vars in 1384 bytes} [Thu May 19 21:37:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 21:37:48.739032 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:37:48.814448 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=6 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:38:03.863150 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d08bbe-0c5d-45b2-bfac-895b9664478d None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:38:03.864721 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f3d08bbe-0c5d-45b2-bfac-895b9664478d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:38:03.880794 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d08bbe-0c5d-45b2-bfac-895b9664478d None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:38:03.886929 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0339ef6-41e0-411f-bf4b-3babfe65ce99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:38:03.887709 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0339ef6-41e0-411f-bf4b-3babfe65ce99 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:38:03.888015 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0339ef6-41e0-411f-bf4b-3babfe65ce99 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:38:03.888635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0339ef6-41e0-411f-bf4b-3babfe65ce99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:38:03.890921 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a0339ef6-41e0-411f-bf4b-3babfe65ce99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:38:03.903772 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0339ef6-41e0-411f-bf4b-3babfe65ce99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:38:03.905500 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0339ef6-41e0-411f-bf4b-3babfe65ce99 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:38:03.916451 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1188/2412] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:38:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:38:05.943540 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a5c73-d92a-4b70-9449-2c2de9c27d3c service swift] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:38:05.956761 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a5c73-d92a-4b70-9449-2c2de9c27d3c service swift] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:38:05.963798 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2241aa67-62dd-40d9-a7d4-d770a40eaf07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:38:05.964509 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241aa67-62dd-40d9-a7d4-d770a40eaf07 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:38:05.965763 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241aa67-62dd-40d9-a7d4-d770a40eaf07 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:38:05.965763 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241aa67-62dd-40d9-a7d4-d770a40eaf07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:38:05.967492 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2241aa67-62dd-40d9-a7d4-d770a40eaf07 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:38:06.034055 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2241aa67-62dd-40d9-a7d4-d770a40eaf07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:38:06.035764 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2241aa67-62dd-40d9-a7d4-d770a40eaf07 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:38:06.050560 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1225/2413] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:38:05 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:38:13.911874 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=6 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:38:16.041933 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=28 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:38:31.962291 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0339ef6-41e0-411f-bf4b-3babfe65ce99 service nova] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:38:31.964214 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a0339ef6-41e0-411f-bf4b-3babfe65ce99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:38:31.981992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0339ef6-41e0-411f-bf4b-3babfe65ce99 service nova] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:38:31.989589 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d0d4a-4775-45f2-89ec-85ecdbbac167 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6143bddc05a46ea972709576c7ad95f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:38:31.990424 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d0d4a-4775-45f2-89ec-85ecdbbac167 service placement] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:38:31.990751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d0d4a-4775-45f2-89ec-85ecdbbac167 service placement] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:38:31.991045 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d0d4a-4775-45f2-89ec-85ecdbbac167 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:38:31.993208 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-568d0d4a-4775-45f2-89ec-85ecdbbac167 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:38:32.009309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568d0d4a-4775-45f2-89ec-85ecdbbac167 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:38:32.011006 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568d0d4a-4775-45f2-89ec-85ecdbbac167 service placement] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:38:32.025031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1189/2414] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:38:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:38:32.036986 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2241aa67-62dd-40d9-a7d4-d770a40eaf07 service nova] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:38:32.037914 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2241aa67-62dd-40d9-a7d4-d770a40eaf07 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:38:32.120270 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2241aa67-62dd-40d9-a7d4-d770a40eaf07 service nova] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:38:32.126151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954e2b1a-4820-4f17-9fa0-41cf99936eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:38:32.127146 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e2b1a-4820-4f17-9fa0-41cf99936eb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:38:32.127578 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e2b1a-4820-4f17-9fa0-41cf99936eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:38:32.127860 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e2b1a-4820-4f17-9fa0-41cf99936eb3 admin admin] PATH_INFO: `/v3/users/1ad8b76baf294f91bc4c641280ab03f7` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:38:32.129200 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954e2b1a-4820-4f17-9fa0-41cf99936eb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ad8b76baf294f91bc4c641280ab03f7)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:38:32.131214 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954e2b1a-4820-4f17-9fa0-41cf99936eb3 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:38:32.188007 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-954e2b1a-4820-4f17-9fa0-41cf99936eb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ad8b76baf294f91bc4c641280ab03f7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:38:32.201736 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-954e2b1a-4820-4f17-9fa0-41cf99936eb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ad8b76baf294f91bc4c641280ab03f7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:38:32.202499 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-954e2b1a-4820-4f17-9fa0-41cf99936eb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ad8b76baf294f91bc4c641280ab03f7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:38:32.203117 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-954e2b1a-4820-4f17-9fa0-41cf99936eb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ad8b76baf294f91bc4c641280ab03f7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:38:32.211041 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-954e2b1a-4820-4f17-9fa0-41cf99936eb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ad8b76baf294f91bc4c641280ab03f7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:38:32.225990 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1226/2415] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:38:32 2022] DELETE /identity/v3/users/1ad8b76baf294f91bc4c641280ab03f7 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:38:32.408999 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d0d4a-4775-45f2-89ec-85ecdbbac167 service placement] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:38:32.420925 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d0d4a-4775-45f2-89ec-85ecdbbac167 service placement] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:38:32.483435 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbd85c4-aa22-4da9-93fc-5a42cbbf5f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:38:32.484311 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd85c4-aa22-4da9-93fc-5a42cbbf5f44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:38:32.484597 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd85c4-aa22-4da9-93fc-5a42cbbf5f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:38:32.484875 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd85c4-aa22-4da9-93fc-5a42cbbf5f44 admin admin] PATH_INFO: `/v3/projects/120a6c484bec4e52bc5891c927647e75` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:38:32.486063 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbd85c4-aa22-4da9-93fc-5a42cbbf5f44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=120a6c484bec4e52bc5891c927647e75)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:38:32.488064 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbd85c4-aa22-4da9-93fc-5a42cbbf5f44 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:38:32.535565 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bbbd85c4-aa22-4da9-93fc-5a42cbbf5f44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '120a6c484bec4e52bc5891c927647e75', 'request_id': 'req-bbbd85c4-aa22-4da9-93fc-5a42cbbf5f44'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:38:32.541697 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bbbd85c4-aa22-4da9-93fc-5a42cbbf5f44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '120a6c484bec4e52bc5891c927647e75', 'request_id': 'req-bbbd85c4-aa22-4da9-93fc-5a42cbbf5f44'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:38:32.550130 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bbbd85c4-aa22-4da9-93fc-5a42cbbf5f44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '120a6c484bec4e52bc5891c927647e75', 'request_id': 'req-bbbd85c4-aa22-4da9-93fc-5a42cbbf5f44'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:38:32.562939 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bbbd85c4-aa22-4da9-93fc-5a42cbbf5f44 admin admin] The token cache is being invalidate because project 120a6c484bec4e52bc5891c927647e75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:38:32.563226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bbbd85c4-aa22-4da9-93fc-5a42cbbf5f44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:38:32.563908 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-bbbd85c4-aa22-4da9-93fc-5a42cbbf5f44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:38:32.565513 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1190/2416] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:38:32 2022] DELETE /identity/v3/projects/120a6c484bec4e52bc5891c927647e75 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:38:42.222172 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:38:42.559022 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:38:43.008426 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36071f-6123-42db-8cc2-bf599b697df5 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:38:43.008713 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36071f-6123-42db-8cc2-bf599b697df5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:38:43.009001 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36071f-6123-42db-8cc2-bf599b697df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:38:43.028352 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-8c36071f-6123-42db-8cc2-bf599b697df5 None None] MFA Rules not processed for user `1589968f375946a38e82ec484afc837f`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:38:43.030331 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8c36071f-6123-42db-8cc2-bf599b697df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:38:43.111876 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1227/2417] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:38:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:38:48.434062 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbd85c4-aa22-4da9-93fc-5a42cbbf5f44 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:38:48.439785 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-bbbd85c4-aa22-4da9-93fc-5a42cbbf5f44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:38:48.550791 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbd85c4-aa22-4da9-93fc-5a42cbbf5f44 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:38:48.563916 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c74ad7a-72c2-4c5d-840c-fe017eb0c57d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:38:48.564751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74ad7a-72c2-4c5d-840c-fe017eb0c57d service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:38:48.565111 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74ad7a-72c2-4c5d-840c-fe017eb0c57d service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:38:48.565448 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74ad7a-72c2-4c5d-840c-fe017eb0c57d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:38:48.568058 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6c74ad7a-72c2-4c5d-840c-fe017eb0c57d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:38:48.582919 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c74ad7a-72c2-4c5d-840c-fe017eb0c57d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:38:48.584954 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c74ad7a-72c2-4c5d-840c-fe017eb0c57d service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:38:48.599023 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1191/2418] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:38:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:38:53.104369 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=25 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:38:58.318035 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c36071f-6123-42db-8cc2-bf599b697df5 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:38:58.332614 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c36071f-6123-42db-8cc2-bf599b697df5 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:38:58.338568 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0237c5ca-a730-4387-92a1-7d94f0a46d14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:38:58.339254 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0237c5ca-a730-4387-92a1-7d94f0a46d14 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:38:58.339543 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0237c5ca-a730-4387-92a1-7d94f0a46d14 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:38:58.339854 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0237c5ca-a730-4387-92a1-7d94f0a46d14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:38:58.342401 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0237c5ca-a730-4387-92a1-7d94f0a46d14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:38:58.415970 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0237c5ca-a730-4387-92a1-7d94f0a46d14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:38:58.417640 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0237c5ca-a730-4387-92a1-7d94f0a46d14 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:38:58.429122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1228/2419] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:38:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:38:58.593376 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=34 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:39:08.423670 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=28 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:39:20.726526 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8433554-7cdd-4c7c-ab2d-e70bd4db6107 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:39:20.727443 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8433554-7cdd-4c7c-ab2d-e70bd4db6107 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:39:20.727443 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8433554-7cdd-4c7c-ab2d-e70bd4db6107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:39:20.758537 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-d8433554-7cdd-4c7c-ab2d-e70bd4db6107 None None] MFA Rules not processed for user `f47e7f746a014cb0a35390e7e0ce435d`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:39:20.759770 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d8433554-7cdd-4c7c-ab2d-e70bd4db6107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:39:20.829411 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1192/2420] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:39:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:39:21.041928 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0237c5ca-a730-4387-92a1-7d94f0a46d14 service nova] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:39:21.043558 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0237c5ca-a730-4387-92a1-7d94f0a46d14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:39:21.059380 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0237c5ca-a730-4387-92a1-7d94f0a46d14 service nova] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:39:21.066312 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0d981-3c9b-4919-8c1e-c547e901036f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:39:21.067280 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0d981-3c9b-4919-8c1e-c547e901036f service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:39:21.067580 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0d981-3c9b-4919-8c1e-c547e901036f service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:39:21.067891 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0d981-3c9b-4919-8c1e-c547e901036f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:39:21.070422 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-04e0d981-3c9b-4919-8c1e-c547e901036f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:39:21.141262 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e0d981-3c9b-4919-8c1e-c547e901036f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:39:21.143181 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e0d981-3c9b-4919-8c1e-c547e901036f service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:39:21.157345 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1229/2421] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:39:21 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:39:21.194195 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8433554-7cdd-4c7c-ab2d-e70bd4db6107 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:39:21.195676 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d8433554-7cdd-4c7c-ab2d-e70bd4db6107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:39:21.264573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8433554-7cdd-4c7c-ab2d-e70bd4db6107 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:39:21.271825 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660c0a5-aff6-48e7-951d-bbc8039bf91d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:39:21.272801 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660c0a5-aff6-48e7-951d-bbc8039bf91d service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:39:21.273261 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660c0a5-aff6-48e7-951d-bbc8039bf91d service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:39:21.273696 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660c0a5-aff6-48e7-951d-bbc8039bf91d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:39:21.278168 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7660c0a5-aff6-48e7-951d-bbc8039bf91d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:39:21.290796 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7660c0a5-aff6-48e7-951d-bbc8039bf91d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:39:21.292101 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7660c0a5-aff6-48e7-951d-bbc8039bf91d service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:39:21.303259 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1193/2422] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:39:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:39:26.433980 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7c718-8ec7-4949-92f9-0eb8c505c965 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:39:26.434737 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7c718-8ec7-4949-92f9-0eb8c505c965 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:39:26.434737 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7c718-8ec7-4949-92f9-0eb8c505c965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:39:26.454153 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-3db7c718-8ec7-4949-92f9-0eb8c505c965 None None] MFA Rules not processed for user `0a58f82117ab437cab5607c204c86547`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:39:26.455393 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3db7c718-8ec7-4949-92f9-0eb8c505c965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:39:26.462913 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1230/2423] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:39:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:39:31.297825 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=51 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:39:32.051243 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660c0a5-aff6-48e7-951d-bbc8039bf91d service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:39:32.066634 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660c0a5-aff6-48e7-951d-bbc8039bf91d service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:39:32.078527 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d10f11f-f13a-4178-b69e-b0967fd272f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:39:32.079175 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10f11f-f13a-4178-b69e-b0967fd272f1 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:39:32.079455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10f11f-f13a-4178-b69e-b0967fd272f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:39:32.079733 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10f11f-f13a-4178-b69e-b0967fd272f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:39:32.082063 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4d10f11f-f13a-4178-b69e-b0967fd272f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:39:32.100741 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d10f11f-f13a-4178-b69e-b0967fd272f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:39:32.103953 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d10f11f-f13a-4178-b69e-b0967fd272f1 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:39:32.120736 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1194/2424] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:39:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:39:36.448008 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=31 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:39:42.116226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=6 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:40:10.721993 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db7c718-8ec7-4949-92f9-0eb8c505c965 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:10.736697 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db7c718-8ec7-4949-92f9-0eb8c505c965 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:10.742964 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5292e9-0fd4-4c3f-9a69-616692b9a8f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:10.743766 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5292e9-0fd4-4c3f-9a69-616692b9a8f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:10.744136 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5292e9-0fd4-4c3f-9a69-616692b9a8f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:10.744417 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5292e9-0fd4-4c3f-9a69-616692b9a8f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:10.746903 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-dc5292e9-0fd4-4c3f-9a69-616692b9a8f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:10.819859 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5292e9-0fd4-4c3f-9a69-616692b9a8f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:10.821281 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5292e9-0fd4-4c3f-9a69-616692b9a8f5 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:10.831986 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1231/2425] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:40:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:40:14.363251 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d10f11f-f13a-4178-b69e-b0967fd272f1 service nova] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:14.364820 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4d10f11f-f13a-4178-b69e-b0967fd272f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:14.388432 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d10f11f-f13a-4178-b69e-b0967fd272f1 service nova] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:14.394322 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86a59d9-d8ea-4fda-9f92-501c93b03452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:14.395225 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a59d9-d8ea-4fda-9f92-501c93b03452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:14.395507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a59d9-d8ea-4fda-9f92-501c93b03452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:14.395784 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a59d9-d8ea-4fda-9f92-501c93b03452 admin admin] PATH_INFO: `/v3/users/1589968f375946a38e82ec484afc837f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:14.397036 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86a59d9-d8ea-4fda-9f92-501c93b03452 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1589968f375946a38e82ec484afc837f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:14.399042 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86a59d9-d8ea-4fda-9f92-501c93b03452 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:14.457830 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d86a59d9-d8ea-4fda-9f92-501c93b03452 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1589968f375946a38e82ec484afc837f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:14.458768 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d86a59d9-d8ea-4fda-9f92-501c93b03452 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1589968f375946a38e82ec484afc837f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:14.473319 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d86a59d9-d8ea-4fda-9f92-501c93b03452 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1589968f375946a38e82ec484afc837f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:14.481831 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d86a59d9-d8ea-4fda-9f92-501c93b03452 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1589968f375946a38e82ec484afc837f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:14.492040 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d86a59d9-d8ea-4fda-9f92-501c93b03452 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1589968f375946a38e82ec484afc837f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:14.500050 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1195/2426] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:40:14 2022] DELETE /identity/v3/users/1589968f375946a38e82ec484afc837f => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:40:14.690008 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5292e9-0fd4-4c3f-9a69-616692b9a8f5 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:14.700664 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5292e9-0fd4-4c3f-9a69-616692b9a8f5 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:14.764998 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce66806-0a9e-46a9-a0f1-a1968fe93924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:14.766047 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce66806-0a9e-46a9-a0f1-a1968fe93924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:14.766370 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce66806-0a9e-46a9-a0f1-a1968fe93924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:14.766651 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce66806-0a9e-46a9-a0f1-a1968fe93924 admin admin] PATH_INFO: `/v3/projects/985c62c1f3c54f568abdac37360ab457` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:14.768033 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce66806-0a9e-46a9-a0f1-a1968fe93924 admin admin] RBAC: Authorizing `identity:delete_project(project_id=985c62c1f3c54f568abdac37360ab457)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:14.770194 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce66806-0a9e-46a9-a0f1-a1968fe93924 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:14.827445 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1ce66806-0a9e-46a9-a0f1-a1968fe93924 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '985c62c1f3c54f568abdac37360ab457', 'request_id': 'req-1ce66806-0a9e-46a9-a0f1-a1968fe93924'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:14.834090 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1ce66806-0a9e-46a9-a0f1-a1968fe93924 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '985c62c1f3c54f568abdac37360ab457', 'request_id': 'req-1ce66806-0a9e-46a9-a0f1-a1968fe93924'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:14.840732 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1ce66806-0a9e-46a9-a0f1-a1968fe93924 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '985c62c1f3c54f568abdac37360ab457', 'request_id': 'req-1ce66806-0a9e-46a9-a0f1-a1968fe93924'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:14.864774 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1ce66806-0a9e-46a9-a0f1-a1968fe93924 admin admin] The token cache is being invalidate because project 985c62c1f3c54f568abdac37360ab457 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:40:14.865128 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1ce66806-0a9e-46a9-a0f1-a1968fe93924 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:14.865726 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1ce66806-0a9e-46a9-a0f1-a1968fe93924 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:14.867596 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1232/2427] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:40:14 2022] DELETE /identity/v3/projects/985c62c1f3c54f568abdac37360ab457 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:40:14.872505 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966837c-a908-453b-8a09-ea4a4caace87 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:14.872796 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966837c-a908-453b-8a09-ea4a4caace87 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:14.873098 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966837c-a908-453b-8a09-ea4a4caace87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:14.892547 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-6966837c-a908-453b-8a09-ea4a4caace87 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:40:14.893819 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6966837c-a908-453b-8a09-ea4a4caace87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:14.971774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1196/2428] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:40:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:40:14.980082 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80c675-2329-4e6f-9b9a-781369ea7f70 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:14.980376 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80c675-2329-4e6f-9b9a-781369ea7f70 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:14.980662 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80c675-2329-4e6f-9b9a-781369ea7f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:15.010418 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-3f80c675-2329-4e6f-9b9a-781369ea7f70 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:40:15.011644 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3f80c675-2329-4e6f-9b9a-781369ea7f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:15.024439 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1233/2429] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:40:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:40:15.028952 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6966837c-a908-453b-8a09-ea4a4caace87 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:15.039987 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6966837c-a908-453b-8a09-ea4a4caace87 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:15.045261 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1642d595-3c61-4a0b-af31-6aee39ed0927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:15.045998 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642d595-3c61-4a0b-af31-6aee39ed0927 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:15.046317 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642d595-3c61-4a0b-af31-6aee39ed0927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:15.046603 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642d595-3c61-4a0b-af31-6aee39ed0927 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:15.047359 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1642d595-3c61-4a0b-af31-6aee39ed0927 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:40:15.047822 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1642d595-3c61-4a0b-af31-6aee39ed0927 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:15.049474 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1642d595-3c61-4a0b-af31-6aee39ed0927 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:15.063414 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1197/2430] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:40:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:40:15.067979 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80c675-2329-4e6f-9b9a-781369ea7f70 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:15.077563 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80c675-2329-4e6f-9b9a-781369ea7f70 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:15.082863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887db261-5498-490e-844d-bd2c3ace1fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:15.083553 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887db261-5498-490e-844d-bd2c3ace1fba admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:15.083842 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887db261-5498-490e-844d-bd2c3ace1fba admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:15.084213 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887db261-5498-490e-844d-bd2c3ace1fba admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:15.084833 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887db261-5498-490e-844d-bd2c3ace1fba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:15.086801 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887db261-5498-490e-844d-bd2c3ace1fba admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:15.102769 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:40:15.140875 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1234/2431] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:40:15 2022] POST /identity/v3/projects => generated 373 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:40:15.142384 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1642d595-3c61-4a0b-af31-6aee39ed0927 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:15.154009 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1642d595-3c61-4a0b-af31-6aee39ed0927 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:15.223283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b76ee98-4d09-4ad4-8672-51fb394459d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:15.224138 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76ee98-4d09-4ad4-8672-51fb394459d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:15.224568 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76ee98-4d09-4ad4-8672-51fb394459d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:15.224940 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76ee98-4d09-4ad4-8672-51fb394459d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:15.225746 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b76ee98-4d09-4ad4-8672-51fb394459d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:15.228439 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b76ee98-4d09-4ad4-8672-51fb394459d4 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:15.236609 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:40:15.254797 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1198/2432] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:40:15 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:40:15.259104 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887db261-5498-490e-844d-bd2c3ace1fba admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:15.270428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887db261-5498-490e-844d-bd2c3ace1fba admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:15.277962 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281a577d-d682-4ab0-aaaa-4d87895511bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:15.279194 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a577d-d682-4ab0-aaaa-4d87895511bd admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:15.279689 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a577d-d682-4ab0-aaaa-4d87895511bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:15.280264 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a577d-d682-4ab0-aaaa-4d87895511bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:15.281643 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281a577d-d682-4ab0-aaaa-4d87895511bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:15.284627 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281a577d-d682-4ab0-aaaa-4d87895511bd admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:15.300474 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1235/2433] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:40:15.305725 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b76ee98-4d09-4ad4-8672-51fb394459d4 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:15.317251 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b76ee98-4d09-4ad4-8672-51fb394459d4 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:15.323636 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c59cd9b-2613-4873-b6cf-3b739c0a6beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:15.324545 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59cd9b-2613-4873-b6cf-3b739c0a6beb admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:15.325147 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59cd9b-2613-4873-b6cf-3b739c0a6beb admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:15.325459 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59cd9b-2613-4873-b6cf-3b739c0a6beb admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:15.326434 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c59cd9b-2613-4873-b6cf-3b739c0a6beb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:15.328823 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c59cd9b-2613-4873-b6cf-3b739c0a6beb admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:15.349561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1199/2434] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:40:15.353878 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281a577d-d682-4ab0-aaaa-4d87895511bd admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:15.365201 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281a577d-d682-4ab0-aaaa-4d87895511bd admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:15.370322 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea1f25-c881-46d4-b224-e61bec6d9078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:15.371241 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea1f25-c881-46d4-b224-e61bec6d9078 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:15.371526 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea1f25-c881-46d4-b224-e61bec6d9078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:15.371805 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea1f25-c881-46d4-b224-e61bec6d9078 admin admin] PATH_INFO: `/v3/projects/c15dc44be00b47b2851f9778d0babe3e/users/03ccb0fbb7ce42c0a71bbd7e604bb743/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:15.388049 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ea1f25-c881-46d4-b224-e61bec6d9078 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c15dc44be00b47b2851f9778d0babe3e, user_id=03ccb0fbb7ce42c0a71bbd7e604bb743, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:15.391140 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ea1f25-c881-46d4-b224-e61bec6d9078 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:15.412977 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1236/2435] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:40:15 2022] PUT /identity/v3/projects/c15dc44be00b47b2851f9778d0babe3e/users/03ccb0fbb7ce42c0a71bbd7e604bb743/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:40:15.430721 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cee6db-7641-4537-9746-798082568c83 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:15.431308 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cee6db-7641-4537-9746-798082568c83 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:15.431308 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cee6db-7641-4537-9746-798082568c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:15.460366 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-e6cee6db-7641-4537-9746-798082568c83 None None] MFA Rules not processed for user `03ccb0fbb7ce42c0a71bbd7e604bb743`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:40:15.461595 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e6cee6db-7641-4537-9746-798082568c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:15.545193 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1200/2436] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:40:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:40:15.557386 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea1f25-c881-46d4-b224-e61bec6d9078 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:15.559331 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e3ea1f25-c881-46d4-b224-e61bec6d9078 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:15.653764 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea1f25-c881-46d4-b224-e61bec6d9078 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:15.662211 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db02de3a-fc05-4d38-9b69-96c480348dbc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:15.662989 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02de3a-fc05-4d38-9b69-96c480348dbc service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:15.663355 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02de3a-fc05-4d38-9b69-96c480348dbc service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:15.663713 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02de3a-fc05-4d38-9b69-96c480348dbc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:15.673085 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db02de3a-fc05-4d38-9b69-96c480348dbc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:15.674837 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db02de3a-fc05-4d38-9b69-96c480348dbc service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:15.685368 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1237/2437] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:40:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:40:15.934201 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cee6db-7641-4537-9746-798082568c83 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:16.023483 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cee6db-7641-4537-9746-798082568c83 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:16.031539 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dabd5f7-9a20-4230-9163-3e0b7087396f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:16.033086 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dabd5f7-9a20-4230-9163-3e0b7087396f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:16.033915 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dabd5f7-9a20-4230-9163-3e0b7087396f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:16.034844 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dabd5f7-9a20-4230-9163-3e0b7087396f admin admin] PATH_INFO: `/v3/users/03ccb0fbb7ce42c0a71bbd7e604bb743` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:16.037681 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dabd5f7-9a20-4230-9163-3e0b7087396f admin admin] RBAC: Authorizing `identity:delete_user(user_id=03ccb0fbb7ce42c0a71bbd7e604bb743)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:16.042069 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dabd5f7-9a20-4230-9163-3e0b7087396f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:16.105385 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8dabd5f7-9a20-4230-9163-3e0b7087396f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03ccb0fbb7ce42c0a71bbd7e604bb743'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:16.106060 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8dabd5f7-9a20-4230-9163-3e0b7087396f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03ccb0fbb7ce42c0a71bbd7e604bb743'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:16.119398 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8dabd5f7-9a20-4230-9163-3e0b7087396f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03ccb0fbb7ce42c0a71bbd7e604bb743'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:16.124379 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:INSERT=3,keystone:SELECT=88,keystone:DELETE=9 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:40:16.128410 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8dabd5f7-9a20-4230-9163-3e0b7087396f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03ccb0fbb7ce42c0a71bbd7e604bb743'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:16.137983 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8dabd5f7-9a20-4230-9163-3e0b7087396f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03ccb0fbb7ce42c0a71bbd7e604bb743'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:16.147120 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1201/2438] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:40:15 2022] DELETE /identity/v3/users/03ccb0fbb7ce42c0a71bbd7e604bb743 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:40:16.151863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4387cf-8e4c-479c-9035-83f01f53e2a0 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:16.152408 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4387cf-8e4c-479c-9035-83f01f53e2a0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:16.152408 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4387cf-8e4c-479c-9035-83f01f53e2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:16.170992 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9f4387cf-8e4c-479c-9035-83f01f53e2a0 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:40:16.172227 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9f4387cf-8e4c-479c-9035-83f01f53e2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:16.241104 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1238/2439] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:40:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:40:16.252151 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dabd5f7-9a20-4230-9163-3e0b7087396f admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:16.253712 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8dabd5f7-9a20-4230-9163-3e0b7087396f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:16.318447 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dabd5f7-9a20-4230-9163-3e0b7087396f admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:16.322420 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6c75a-71d0-4cc8-98ca-91b43ed5011c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:16.323134 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6c75a-71d0-4cc8-98ca-91b43ed5011c service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:16.323455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6c75a-71d0-4cc8-98ca-91b43ed5011c service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:16.323770 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6c75a-71d0-4cc8-98ca-91b43ed5011c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:16.335697 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad6c75a-71d0-4cc8-98ca-91b43ed5011c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:16.337176 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad6c75a-71d0-4cc8-98ca-91b43ed5011c service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:16.348144 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1202/2440] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:40:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:40:16.684886 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4387cf-8e4c-479c-9035-83f01f53e2a0 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:16.687321 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9f4387cf-8e4c-479c-9035-83f01f53e2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:16.707278 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4387cf-8e4c-479c-9035-83f01f53e2a0 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:16.714858 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae91f06b-6f2a-4840-a03c-a3a3fdc2497d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:16.716476 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae91f06b-6f2a-4840-a03c-a3a3fdc2497d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:16.716754 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae91f06b-6f2a-4840-a03c-a3a3fdc2497d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:16.717038 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae91f06b-6f2a-4840-a03c-a3a3fdc2497d admin admin] PATH_INFO: `/v3/projects/c15dc44be00b47b2851f9778d0babe3e` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:16.719001 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae91f06b-6f2a-4840-a03c-a3a3fdc2497d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c15dc44be00b47b2851f9778d0babe3e)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:16.720820 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae91f06b-6f2a-4840-a03c-a3a3fdc2497d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:16.761965 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:40:16.776491 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ae91f06b-6f2a-4840-a03c-a3a3fdc2497d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c15dc44be00b47b2851f9778d0babe3e', 'request_id': 'req-ae91f06b-6f2a-4840-a03c-a3a3fdc2497d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:16.783245 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ae91f06b-6f2a-4840-a03c-a3a3fdc2497d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c15dc44be00b47b2851f9778d0babe3e', 'request_id': 'req-ae91f06b-6f2a-4840-a03c-a3a3fdc2497d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:16.789036 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ae91f06b-6f2a-4840-a03c-a3a3fdc2497d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c15dc44be00b47b2851f9778d0babe3e', 'request_id': 'req-ae91f06b-6f2a-4840-a03c-a3a3fdc2497d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:16.803733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ae91f06b-6f2a-4840-a03c-a3a3fdc2497d admin admin] The token cache is being invalidate because project c15dc44be00b47b2851f9778d0babe3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:40:16.804058 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ae91f06b-6f2a-4840-a03c-a3a3fdc2497d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:16.804776 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ae91f06b-6f2a-4840-a03c-a3a3fdc2497d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:16.806568 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1239/2441] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:40:16 2022] DELETE /identity/v3/projects/c15dc44be00b47b2851f9778d0babe3e => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:40:16.812836 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761eb8c-a990-48b3-859b-aa7c4e291bed None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:16.813394 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761eb8c-a990-48b3-859b-aa7c4e291bed None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:16.813394 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761eb8c-a990-48b3-859b-aa7c4e291bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:16.832440 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-2761eb8c-a990-48b3-859b-aa7c4e291bed None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:40:16.833743 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-2761eb8c-a990-48b3-859b-aa7c4e291bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:16.910446 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1203/2442] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:40:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:40:16.919836 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91eb90-b9d5-4b90-b78e-7758e242755f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:16.920151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91eb90-b9d5-4b90-b78e-7758e242755f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:16.920566 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91eb90-b9d5-4b90-b78e-7758e242755f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:16.942765 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-6a91eb90-b9d5-4b90-b78e-7758e242755f None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:40:16.944051 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6a91eb90-b9d5-4b90-b78e-7758e242755f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:16.952382 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1240/2443] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:40:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:40:16.957315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2761eb8c-a990-48b3-859b-aa7c4e291bed None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:16.968525 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2761eb8c-a990-48b3-859b-aa7c4e291bed None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:16.973699 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b857d6f9-7a25-461c-9cb6-732fcd5cd04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:16.974425 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857d6f9-7a25-461c-9cb6-732fcd5cd04b admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:16.974717 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857d6f9-7a25-461c-9cb6-732fcd5cd04b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:16.974990 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857d6f9-7a25-461c-9cb6-732fcd5cd04b admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:16.975733 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b857d6f9-7a25-461c-9cb6-732fcd5cd04b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:40:16.976036 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b857d6f9-7a25-461c-9cb6-732fcd5cd04b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:16.977667 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b857d6f9-7a25-461c-9cb6-732fcd5cd04b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:16.991063 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1204/2444] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:40:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:40:16.995880 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a91eb90-b9d5-4b90-b78e-7758e242755f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:17.006372 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a91eb90-b9d5-4b90-b78e-7758e242755f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:17.012236 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b930f39-7187-4c23-9533-93e973de896b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:17.012966 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b930f39-7187-4c23-9533-93e973de896b admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:17.013288 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b930f39-7187-4c23-9533-93e973de896b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:17.013620 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b930f39-7187-4c23-9533-93e973de896b admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:17.014763 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b930f39-7187-4c23-9533-93e973de896b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:17.017181 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b930f39-7187-4c23-9533-93e973de896b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:17.049728 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1241/2445] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:40:16 2022] POST /identity/v3/projects => generated 365 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:40:17.054416 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b857d6f9-7a25-461c-9cb6-732fcd5cd04b admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:17.065394 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b857d6f9-7a25-461c-9cb6-732fcd5cd04b admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:17.129089 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb58399-7946-4b82-b479-68f9e28d536c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:17.129784 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb58399-7946-4b82-b479-68f9e28d536c admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:17.130065 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb58399-7946-4b82-b479-68f9e28d536c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:17.130400 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb58399-7946-4b82-b479-68f9e28d536c admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:17.131115 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb58399-7946-4b82-b479-68f9e28d536c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:17.133131 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb58399-7946-4b82-b479-68f9e28d536c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:17.159410 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1205/2446] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:40:17 2022] POST /identity/v3/users => generated 283 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:40:17.165108 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b930f39-7187-4c23-9533-93e973de896b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:17.179055 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b930f39-7187-4c23-9533-93e973de896b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:17.184281 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f127b-86b0-4ae3-97e3-f736ba33dea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:17.184971 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f127b-86b0-4ae3-97e3-f736ba33dea8 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:17.185253 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f127b-86b0-4ae3-97e3-f736ba33dea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:17.185526 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f127b-86b0-4ae3-97e3-f736ba33dea8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:17.186410 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1f127b-86b0-4ae3-97e3-f736ba33dea8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:17.188079 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1f127b-86b0-4ae3-97e3-f736ba33dea8 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:17.199652 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1242/2447] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:40:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:40:17.205634 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb58399-7946-4b82-b479-68f9e28d536c admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:17.217108 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb58399-7946-4b82-b479-68f9e28d536c admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:17.227358 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea55b4e2-db95-44ae-97cd-ccc99e27141e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:17.228045 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55b4e2-db95-44ae-97cd-ccc99e27141e admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:17.228337 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55b4e2-db95-44ae-97cd-ccc99e27141e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:17.228630 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55b4e2-db95-44ae-97cd-ccc99e27141e admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:17.229402 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea55b4e2-db95-44ae-97cd-ccc99e27141e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:17.231140 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea55b4e2-db95-44ae-97cd-ccc99e27141e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:17.232696 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:40:17.245348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1206/2448] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:40:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:40:17.250283 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f127b-86b0-4ae3-97e3-f736ba33dea8 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:17.261889 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f127b-86b0-4ae3-97e3-f736ba33dea8 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:17.267608 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484918de-a73e-4249-a5d9-02b9b5ead75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:17.268528 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484918de-a73e-4249-a5d9-02b9b5ead75c admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:17.268809 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484918de-a73e-4249-a5d9-02b9b5ead75c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:17.269094 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484918de-a73e-4249-a5d9-02b9b5ead75c admin admin] PATH_INFO: `/v3/projects/9fefd188a7dc47b49d44197f6a074665/users/ae07acce51cf4d288cf484a11c382cc3/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:17.286467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484918de-a73e-4249-a5d9-02b9b5ead75c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fefd188a7dc47b49d44197f6a074665, user_id=ae07acce51cf4d288cf484a11c382cc3, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:17.289156 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484918de-a73e-4249-a5d9-02b9b5ead75c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:17.304153 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1243/2449] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:40:17 2022] PUT /identity/v3/projects/9fefd188a7dc47b49d44197f6a074665/users/ae07acce51cf4d288cf484a11c382cc3/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:40:17.316218 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c878e-843a-4f89-812b-def808547aee None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:17.317428 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c878e-843a-4f89-812b-def808547aee None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:17.317582 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c878e-843a-4f89-812b-def808547aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:17.347342 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-788c878e-843a-4f89-812b-def808547aee None None] MFA Rules not processed for user `ae07acce51cf4d288cf484a11c382cc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:40:17.348578 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-788c878e-843a-4f89-812b-def808547aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:17.420191 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1207/2450] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:40:17 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:40:17.425406 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484918de-a73e-4249-a5d9-02b9b5ead75c admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:17.503622 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484918de-a73e-4249-a5d9-02b9b5ead75c admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:17.509154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbfa2d2-5f63-4d91-bd1f-ce7ab55f5520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:17.509833 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbfa2d2-5f63-4d91-bd1f-ce7ab55f5520 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:17.510120 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbfa2d2-5f63-4d91-bd1f-ce7ab55f5520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:17.510434 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbfa2d2-5f63-4d91-bd1f-ce7ab55f5520 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:17.511132 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbfa2d2-5f63-4d91-bd1f-ce7ab55f5520 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:17.513062 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbfa2d2-5f63-4d91-bd1f-ce7ab55f5520 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:17.542373 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1244/2451] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:40:17 2022] POST /identity/v3/projects => generated 363 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:40:17.547558 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c878e-843a-4f89-812b-def808547aee None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:17.559398 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c878e-843a-4f89-812b-def808547aee None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:17.622154 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e831d-8d26-420b-b5db-4e6cdd71d585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:17.622908 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e831d-8d26-420b-b5db-4e6cdd71d585 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:17.623195 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e831d-8d26-420b-b5db-4e6cdd71d585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:17.623987 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e831d-8d26-420b-b5db-4e6cdd71d585 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:17.624706 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6e831d-8d26-420b-b5db-4e6cdd71d585 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:17.626738 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6e831d-8d26-420b-b5db-4e6cdd71d585 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:17.649786 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1208/2452] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:40:17 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:40:17.654435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbfa2d2-5f63-4d91-bd1f-ce7ab55f5520 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:17.666918 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbfa2d2-5f63-4d91-bd1f-ce7ab55f5520 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:17.672922 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0771ea2-ee6f-4218-a2b0-7e7fccda9688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:17.673546 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0771ea2-ee6f-4218-a2b0-7e7fccda9688 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:17.673832 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0771ea2-ee6f-4218-a2b0-7e7fccda9688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:17.674113 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0771ea2-ee6f-4218-a2b0-7e7fccda9688 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:17.674898 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0771ea2-ee6f-4218-a2b0-7e7fccda9688 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:17.676513 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0771ea2-ee6f-4218-a2b0-7e7fccda9688 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:17.687954 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1245/2453] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:40:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:40:17.692838 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e831d-8d26-420b-b5db-4e6cdd71d585 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:17.704806 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e831d-8d26-420b-b5db-4e6cdd71d585 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:17.710141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a52bde-171a-4732-8261-02fdca7547b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:17.711066 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a52bde-171a-4732-8261-02fdca7547b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:17.711363 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a52bde-171a-4732-8261-02fdca7547b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:17.711641 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a52bde-171a-4732-8261-02fdca7547b8 admin admin] PATH_INFO: `/v3/projects/40c6078f031f4ce1b2a80535fb174ede/users/5f816c521178459c9c5bc7b33dc750a4/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:17.730339 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a52bde-171a-4732-8261-02fdca7547b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40c6078f031f4ce1b2a80535fb174ede, user_id=5f816c521178459c9c5bc7b33dc750a4, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:17.733278 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a52bde-171a-4732-8261-02fdca7547b8 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:17.751552 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1209/2454] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:40:17 2022] PUT /identity/v3/projects/40c6078f031f4ce1b2a80535fb174ede/users/5f816c521178459c9c5bc7b33dc750a4/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:40:17.756183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0771ea2-ee6f-4218-a2b0-7e7fccda9688 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:17.767792 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0771ea2-ee6f-4218-a2b0-7e7fccda9688 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:17.801076 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:40:17.836361 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc11a5ad-c7fb-4e52-8248-4bc1a866b461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:17.837058 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc11a5ad-c7fb-4e52-8248-4bc1a866b461 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:17.837338 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc11a5ad-c7fb-4e52-8248-4bc1a866b461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:17.837614 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc11a5ad-c7fb-4e52-8248-4bc1a866b461 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:17.838382 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc11a5ad-c7fb-4e52-8248-4bc1a866b461 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:17.840118 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc11a5ad-c7fb-4e52-8248-4bc1a866b461 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:17.853280 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1246/2455] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:40:17 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:40:17.857728 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a52bde-171a-4732-8261-02fdca7547b8 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:17.872966 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a52bde-171a-4732-8261-02fdca7547b8 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:17.881761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b6f15e-767b-45a0-9725-abfc0369cecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:17.883361 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b6f15e-767b-45a0-9725-abfc0369cecc admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:17.883843 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b6f15e-767b-45a0-9725-abfc0369cecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:17.884334 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b6f15e-767b-45a0-9725-abfc0369cecc admin admin] PATH_INFO: `/v3/projects/40c6078f031f4ce1b2a80535fb174ede/users/5f816c521178459c9c5bc7b33dc750a4/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:17.888036 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b6f15e-767b-45a0-9725-abfc0369cecc admin admin] RBAC: Authorizing `identity:create_grant(project_id=40c6078f031f4ce1b2a80535fb174ede, user_id=5f816c521178459c9c5bc7b33dc750a4, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:17.893954 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b6f15e-767b-45a0-9725-abfc0369cecc admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:17.913875 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1210/2456] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:40:17 2022] PUT /identity/v3/projects/40c6078f031f4ce1b2a80535fb174ede/users/5f816c521178459c9c5bc7b33dc750a4/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:40:17.927685 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf34de-bb74-4776-af45-e0ed07cee61d None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:17.927900 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf34de-bb74-4776-af45-e0ed07cee61d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:17.928150 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf34de-bb74-4776-af45-e0ed07cee61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:17.959996 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-3baf34de-bb74-4776-af45-e0ed07cee61d None None] MFA Rules not processed for user `5f816c521178459c9c5bc7b33dc750a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:40:17.961253 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3baf34de-bb74-4776-af45-e0ed07cee61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:18.066458 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1247/2457] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:40:17 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:40:18.086954 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b6f15e-767b-45a0-9725-abfc0369cecc admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:18.089720 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-28b6f15e-767b-45a0-9725-abfc0369cecc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:18.160017 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:40:18.180438 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b6f15e-767b-45a0-9725-abfc0369cecc admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:18.184208 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b152e2a-291b-405b-9304-72dc3eebbb2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:18.184895 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b152e2a-291b-405b-9304-72dc3eebbb2c service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:18.185183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b152e2a-291b-405b-9304-72dc3eebbb2c service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:18.185483 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b152e2a-291b-405b-9304-72dc3eebbb2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:18.196200 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b152e2a-291b-405b-9304-72dc3eebbb2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:18.197651 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b152e2a-291b-405b-9304-72dc3eebbb2c service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:18.208487 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1211/2458] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:40:18 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:40:18.307912 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baf34de-bb74-4776-af45-e0ed07cee61d None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:18.321916 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baf34de-bb74-4776-af45-e0ed07cee61d None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:18.326088 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d957feb-0843-4d84-af99-0b8311fc4e96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:18.327157 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d957feb-0843-4d84-af99-0b8311fc4e96 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:18.327421 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d957feb-0843-4d84-af99-0b8311fc4e96 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:18.327704 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d957feb-0843-4d84-af99-0b8311fc4e96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:18.394853 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d957feb-0843-4d84-af99-0b8311fc4e96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:18.396334 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d957feb-0843-4d84-af99-0b8311fc4e96 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:18.407501 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1248/2459] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:40:18 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:40:27.559709 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b152e2a-291b-405b-9304-72dc3eebbb2c service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:27.560890 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5b152e2a-291b-405b-9304-72dc3eebbb2c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:27.655898 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b152e2a-291b-405b-9304-72dc3eebbb2c service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:27.661957 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8ce64-2ec4-4e36-b23b-b9c1f2eb18e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:27.662669 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8ce64-2ec4-4e36-b23b-b9c1f2eb18e7 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:27.662954 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8ce64-2ec4-4e36-b23b-b9c1f2eb18e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:27.663232 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8ce64-2ec4-4e36-b23b-b9c1f2eb18e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:27.665302 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f2b8ce64-2ec4-4e36-b23b-b9c1f2eb18e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:27.737996 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b8ce64-2ec4-4e36-b23b-b9c1f2eb18e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:27.740681 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b8ce64-2ec4-4e36-b23b-b9c1f2eb18e7 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:27.755321 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1212/2460] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:40:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:40:28.402088 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=77 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:40:30.285837 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d957feb-0843-4d84-af99-0b8311fc4e96 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:30.366668 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d957feb-0843-4d84-af99-0b8311fc4e96 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:30.371935 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2186be14-4997-4a94-a09a-34080ee3189b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:30.372838 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186be14-4997-4a94-a09a-34080ee3189b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:30.373189 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186be14-4997-4a94-a09a-34080ee3189b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:30.373472 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186be14-4997-4a94-a09a-34080ee3189b admin admin] PATH_INFO: `/v3/users/ae07acce51cf4d288cf484a11c382cc3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:30.374752 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2186be14-4997-4a94-a09a-34080ee3189b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae07acce51cf4d288cf484a11c382cc3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:30.376850 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2186be14-4997-4a94-a09a-34080ee3189b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:30.436947 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2186be14-4997-4a94-a09a-34080ee3189b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae07acce51cf4d288cf484a11c382cc3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:30.450130 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2186be14-4997-4a94-a09a-34080ee3189b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae07acce51cf4d288cf484a11c382cc3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:30.450886 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2186be14-4997-4a94-a09a-34080ee3189b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae07acce51cf4d288cf484a11c382cc3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:30.451670 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2186be14-4997-4a94-a09a-34080ee3189b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae07acce51cf4d288cf484a11c382cc3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:30.460602 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2186be14-4997-4a94-a09a-34080ee3189b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae07acce51cf4d288cf484a11c382cc3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:30.482967 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1249/2461] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:40:30 2022] DELETE /identity/v3/users/ae07acce51cf4d288cf484a11c382cc3 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:40:30.487163 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc726a9-3bc8-405f-86eb-3ffe36fa8633 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:30.487664 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc726a9-3bc8-405f-86eb-3ffe36fa8633 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:30.487910 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc726a9-3bc8-405f-86eb-3ffe36fa8633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:30.506848 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-acc726a9-3bc8-405f-86eb-3ffe36fa8633 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:40:30.508135 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-acc726a9-3bc8-405f-86eb-3ffe36fa8633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:30.633994 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1213/2462] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:40:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:40:30.649164 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2186be14-4997-4a94-a09a-34080ee3189b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:30.651451 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2186be14-4997-4a94-a09a-34080ee3189b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:30.749527 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2186be14-4997-4a94-a09a-34080ee3189b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:30.753843 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e59c4-f9f3-4376-a73f-7aabe1256ec6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:30.754800 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e59c4-f9f3-4376-a73f-7aabe1256ec6 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:30.755152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e59c4-f9f3-4376-a73f-7aabe1256ec6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:30.755493 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e59c4-f9f3-4376-a73f-7aabe1256ec6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:30.781232 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738e59c4-f9f3-4376-a73f-7aabe1256ec6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:30.782769 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738e59c4-f9f3-4376-a73f-7aabe1256ec6 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:30.807372 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1250/2463] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:40:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:40:31.284657 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc726a9-3bc8-405f-86eb-3ffe36fa8633 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:31.286567 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-acc726a9-3bc8-405f-86eb-3ffe36fa8633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:31.309611 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc726a9-3bc8-405f-86eb-3ffe36fa8633 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:31.317213 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449414e-da48-41cd-92ff-3d9acbacde4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:31.322957 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449414e-da48-41cd-92ff-3d9acbacde4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:31.323176 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449414e-da48-41cd-92ff-3d9acbacde4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:31.323456 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449414e-da48-41cd-92ff-3d9acbacde4f admin admin] PATH_INFO: `/v3/projects/9fefd188a7dc47b49d44197f6a074665` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:31.324900 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b449414e-da48-41cd-92ff-3d9acbacde4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fefd188a7dc47b49d44197f6a074665)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:31.328632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b449414e-da48-41cd-92ff-3d9acbacde4f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:31.347996 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:40:31.399030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b449414e-da48-41cd-92ff-3d9acbacde4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fefd188a7dc47b49d44197f6a074665', 'request_id': 'req-b449414e-da48-41cd-92ff-3d9acbacde4f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:31.406422 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b449414e-da48-41cd-92ff-3d9acbacde4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fefd188a7dc47b49d44197f6a074665', 'request_id': 'req-b449414e-da48-41cd-92ff-3d9acbacde4f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:31.415826 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b449414e-da48-41cd-92ff-3d9acbacde4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fefd188a7dc47b49d44197f6a074665', 'request_id': 'req-b449414e-da48-41cd-92ff-3d9acbacde4f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:31.428011 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b449414e-da48-41cd-92ff-3d9acbacde4f admin admin] The token cache is being invalidate because project 9fefd188a7dc47b49d44197f6a074665 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:40:31.428300 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b449414e-da48-41cd-92ff-3d9acbacde4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:31.429055 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b449414e-da48-41cd-92ff-3d9acbacde4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:31.430623 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1214/2464] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:40:31 2022] DELETE /identity/v3/projects/9fefd188a7dc47b49d44197f6a074665 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:40:31.434701 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e59c4-f9f3-4376-a73f-7aabe1256ec6 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:31.444612 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e59c4-f9f3-4376-a73f-7aabe1256ec6 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:31.468552 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:40:31.513050 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38aa257-57d0-4d1a-9ff9-2aebbc965d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:31.514280 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38aa257-57d0-4d1a-9ff9-2aebbc965d8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:31.514588 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38aa257-57d0-4d1a-9ff9-2aebbc965d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:31.514860 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38aa257-57d0-4d1a-9ff9-2aebbc965d8e admin admin] PATH_INFO: `/v3/users/5f816c521178459c9c5bc7b33dc750a4` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:31.516104 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38aa257-57d0-4d1a-9ff9-2aebbc965d8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f816c521178459c9c5bc7b33dc750a4)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:31.518035 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38aa257-57d0-4d1a-9ff9-2aebbc965d8e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:31.571510 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a38aa257-57d0-4d1a-9ff9-2aebbc965d8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f816c521178459c9c5bc7b33dc750a4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:31.586459 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a38aa257-57d0-4d1a-9ff9-2aebbc965d8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f816c521178459c9c5bc7b33dc750a4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:31.587193 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a38aa257-57d0-4d1a-9ff9-2aebbc965d8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f816c521178459c9c5bc7b33dc750a4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:31.587779 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a38aa257-57d0-4d1a-9ff9-2aebbc965d8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f816c521178459c9c5bc7b33dc750a4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:31.595523 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a38aa257-57d0-4d1a-9ff9-2aebbc965d8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f816c521178459c9c5bc7b33dc750a4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:31.609970 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1251/2465] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:40:31 2022] DELETE /identity/v3/users/5f816c521178459c9c5bc7b33dc750a4 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:40:32.129229 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449414e-da48-41cd-92ff-3d9acbacde4f admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:32.131040 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b449414e-da48-41cd-92ff-3d9acbacde4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:32.161145 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449414e-da48-41cd-92ff-3d9acbacde4f admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:32.252583 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3fda80-6a1d-402a-8805-9bf93c3bfcef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:32.254128 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3fda80-6a1d-402a-8805-9bf93c3bfcef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:32.254690 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3fda80-6a1d-402a-8805-9bf93c3bfcef admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:32.255172 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3fda80-6a1d-402a-8805-9bf93c3bfcef admin admin] PATH_INFO: `/v3/projects/40c6078f031f4ce1b2a80535fb174ede` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:32.257419 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3fda80-6a1d-402a-8805-9bf93c3bfcef admin admin] RBAC: Authorizing `identity:delete_project(project_id=40c6078f031f4ce1b2a80535fb174ede)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:32.261027 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3fda80-6a1d-402a-8805-9bf93c3bfcef admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:32.337231 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-fd3fda80-6a1d-402a-8805-9bf93c3bfcef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40c6078f031f4ce1b2a80535fb174ede', 'request_id': 'req-fd3fda80-6a1d-402a-8805-9bf93c3bfcef'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:32.345153 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-fd3fda80-6a1d-402a-8805-9bf93c3bfcef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40c6078f031f4ce1b2a80535fb174ede', 'request_id': 'req-fd3fda80-6a1d-402a-8805-9bf93c3bfcef'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:32.355313 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-fd3fda80-6a1d-402a-8805-9bf93c3bfcef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40c6078f031f4ce1b2a80535fb174ede', 'request_id': 'req-fd3fda80-6a1d-402a-8805-9bf93c3bfcef'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:32.369933 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-fd3fda80-6a1d-402a-8805-9bf93c3bfcef admin admin] The token cache is being invalidate because project 40c6078f031f4ce1b2a80535fb174ede was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:40:32.370717 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-fd3fda80-6a1d-402a-8805-9bf93c3bfcef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:32.371929 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-fd3fda80-6a1d-402a-8805-9bf93c3bfcef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:32.377450 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1215/2466] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:40:32 2022] DELETE /identity/v3/projects/40c6078f031f4ce1b2a80535fb174ede => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:40:32.379437 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0d516-2a16-4c03-9f86-2fcaf01ca16a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:32.379743 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0d516-2a16-4c03-9f86-2fcaf01ca16a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:32.380028 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0d516-2a16-4c03-9f86-2fcaf01ca16a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:32.417494 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-1fa0d516-2a16-4c03-9f86-2fcaf01ca16a None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:40:32.420835 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1fa0d516-2a16-4c03-9f86-2fcaf01ca16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:32.498925 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1252/2467] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:40:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:40:32.507489 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368f4ee-0524-47aa-8884-753e459f3ac8 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:32.508840 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368f4ee-0524-47aa-8884-753e459f3ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:32.509389 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368f4ee-0524-47aa-8884-753e459f3ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:32.535235 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-a368f4ee-0524-47aa-8884-753e459f3ac8 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:40:32.536743 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a368f4ee-0524-47aa-8884-753e459f3ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:32.554983 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1216/2468] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:40:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:40:32.557158 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa0d516-2a16-4c03-9f86-2fcaf01ca16a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:32.571138 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa0d516-2a16-4c03-9f86-2fcaf01ca16a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:32.578445 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769296aa-5da2-467b-864b-a67d5ac33374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:32.579139 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769296aa-5da2-467b-864b-a67d5ac33374 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:32.579425 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769296aa-5da2-467b-864b-a67d5ac33374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:32.579687 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769296aa-5da2-467b-864b-a67d5ac33374 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:32.580435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769296aa-5da2-467b-864b-a67d5ac33374 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:40:32.580735 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769296aa-5da2-467b-864b-a67d5ac33374 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:32.588198 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769296aa-5da2-467b-864b-a67d5ac33374 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:32.608833 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1253/2469] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:40:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:40:32.610229 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a368f4ee-0524-47aa-8884-753e459f3ac8 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:32.625189 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a368f4ee-0524-47aa-8884-753e459f3ac8 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:32.635682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1493cccb-cf3a-4779-bc6f-904ea59d948a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:32.637396 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493cccb-cf3a-4779-bc6f-904ea59d948a admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:32.638716 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493cccb-cf3a-4779-bc6f-904ea59d948a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:32.639562 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493cccb-cf3a-4779-bc6f-904ea59d948a admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:32.641023 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1493cccb-cf3a-4779-bc6f-904ea59d948a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:32.645542 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1493cccb-cf3a-4779-bc6f-904ea59d948a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:32.698769 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1217/2470] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:40:32 2022] POST /identity/v3/projects => generated 389 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:40:32.702010 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769296aa-5da2-467b-864b-a67d5ac33374 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:32.713903 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769296aa-5da2-467b-864b-a67d5ac33374 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:32.746797 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:40:32.787306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee181a0-2677-47c8-8bbd-f8d857693f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:32.788028 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee181a0-2677-47c8-8bbd-f8d857693f97 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:32.788359 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee181a0-2677-47c8-8bbd-f8d857693f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:32.788733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee181a0-2677-47c8-8bbd-f8d857693f97 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:32.789444 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee181a0-2677-47c8-8bbd-f8d857693f97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:32.791406 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee181a0-2677-47c8-8bbd-f8d857693f97 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:32.813959 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1254/2471] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:40:32 2022] POST /identity/v3/users => generated 295 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:40:32.819380 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1493cccb-cf3a-4779-bc6f-904ea59d948a admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:32.825056 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:40:32.832149 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1493cccb-cf3a-4779-bc6f-904ea59d948a admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:32.838797 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b268527-05d9-49f9-a90f-342693aeac9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:32.839618 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b268527-05d9-49f9-a90f-342693aeac9f admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:32.840031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b268527-05d9-49f9-a90f-342693aeac9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:32.840511 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b268527-05d9-49f9-a90f-342693aeac9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:32.841269 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b268527-05d9-49f9-a90f-342693aeac9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:32.843028 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b268527-05d9-49f9-a90f-342693aeac9f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:32.858644 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1218/2472] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:40:32.860199 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee181a0-2677-47c8-8bbd-f8d857693f97 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:32.873562 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee181a0-2677-47c8-8bbd-f8d857693f97 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:32.878365 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510037a7-134c-4148-8c38-a5ccec0331e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:32.879054 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510037a7-134c-4148-8c38-a5ccec0331e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:32.879341 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510037a7-134c-4148-8c38-a5ccec0331e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:32.879622 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510037a7-134c-4148-8c38-a5ccec0331e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:32.880366 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510037a7-134c-4148-8c38-a5ccec0331e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:32.881994 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510037a7-134c-4148-8c38-a5ccec0331e9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:32.895298 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1255/2473] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:40:32.900091 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b268527-05d9-49f9-a90f-342693aeac9f admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:32.911554 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b268527-05d9-49f9-a90f-342693aeac9f admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:32.916624 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b11db-3807-43fa-8c15-f34e9097cf2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:32.917598 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b11db-3807-43fa-8c15-f34e9097cf2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:32.917880 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b11db-3807-43fa-8c15-f34e9097cf2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:32.918156 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b11db-3807-43fa-8c15-f34e9097cf2b admin admin] PATH_INFO: `/v3/projects/14328cf4dee4416e83699f945c365049/users/5a274fb66ff64d2e88232c2b6ea3e013/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:32.941874 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80b11db-3807-43fa-8c15-f34e9097cf2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=14328cf4dee4416e83699f945c365049, user_id=5a274fb66ff64d2e88232c2b6ea3e013, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:32.945179 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80b11db-3807-43fa-8c15-f34e9097cf2b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:32.963389 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1219/2474] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:40:32 2022] PUT /identity/v3/projects/14328cf4dee4416e83699f945c365049/users/5a274fb66ff64d2e88232c2b6ea3e013/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:40:32.978321 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31666cc-cdd6-4a2b-8ec7-750d90ef9921 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:32.978729 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31666cc-cdd6-4a2b-8ec7-750d90ef9921 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:32.979013 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31666cc-cdd6-4a2b-8ec7-750d90ef9921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:33.030930 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f31666cc-cdd6-4a2b-8ec7-750d90ef9921 None None] MFA Rules not processed for user `5a274fb66ff64d2e88232c2b6ea3e013`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:40:33.032500 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f31666cc-cdd6-4a2b-8ec7-750d90ef9921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:33.111830 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1256/2475] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:40:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:40:33.116794 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b11db-3807-43fa-8c15-f34e9097cf2b admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:33.181950 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b11db-3807-43fa-8c15-f34e9097cf2b admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:33.187399 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a11a130-aad0-4109-9f14-7fe4ada53f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:33.188068 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a11a130-aad0-4109-9f14-7fe4ada53f3f admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:33.188365 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a11a130-aad0-4109-9f14-7fe4ada53f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:33.188636 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a11a130-aad0-4109-9f14-7fe4ada53f3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:33.189331 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a11a130-aad0-4109-9f14-7fe4ada53f3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:33.191301 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a11a130-aad0-4109-9f14-7fe4ada53f3f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:33.221460 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1220/2476] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:40:33 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:40:33.225692 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31666cc-cdd6-4a2b-8ec7-750d90ef9921 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:33.240052 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31666cc-cdd6-4a2b-8ec7-750d90ef9921 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:33.319071 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eedcb2-dfc7-4496-a8a8-13859500e1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:33.320435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eedcb2-dfc7-4496-a8a8-13859500e1c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:33.320815 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eedcb2-dfc7-4496-a8a8-13859500e1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:33.321193 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eedcb2-dfc7-4496-a8a8-13859500e1c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:33.322120 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eedcb2-dfc7-4496-a8a8-13859500e1c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:33.324387 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eedcb2-dfc7-4496-a8a8-13859500e1c8 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:33.370772 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1257/2477] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:40:33 2022] POST /identity/v3/users => generated 300 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:40:33.371806 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a11a130-aad0-4109-9f14-7fe4ada53f3f admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:33.391447 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a11a130-aad0-4109-9f14-7fe4ada53f3f admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:33.397061 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec05da-d27f-4ef3-8fba-ff826b82cbcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:33.398054 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec05da-d27f-4ef3-8fba-ff826b82cbcb admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:33.398054 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec05da-d27f-4ef3-8fba-ff826b82cbcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:33.399738 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec05da-d27f-4ef3-8fba-ff826b82cbcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:33.400514 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ec05da-d27f-4ef3-8fba-ff826b82cbcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:33.402187 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ec05da-d27f-4ef3-8fba-ff826b82cbcb admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:33.427362 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1221/2478] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 21:40:33.427798 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eedcb2-dfc7-4496-a8a8-13859500e1c8 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:33.449433 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eedcb2-dfc7-4496-a8a8-13859500e1c8 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:33.458057 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266fdbc0-4135-49cf-8a1f-13f6b0d404a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:33.459336 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266fdbc0-4135-49cf-8a1f-13f6b0d404a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:33.459695 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266fdbc0-4135-49cf-8a1f-13f6b0d404a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:33.460065 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266fdbc0-4135-49cf-8a1f-13f6b0d404a1 admin admin] PATH_INFO: `/v3/projects/7a9fe07c0f9b4a71b8dbf3a51919c9b8/users/d2b37d4e1c4c4b7a9b7791e091d4e3cf/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:33.486514 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266fdbc0-4135-49cf-8a1f-13f6b0d404a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a9fe07c0f9b4a71b8dbf3a51919c9b8, user_id=d2b37d4e1c4c4b7a9b7791e091d4e3cf, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:33.496005 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec05da-d27f-4ef3-8fba-ff826b82cbcb admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:33.496923 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266fdbc0-4135-49cf-8a1f-13f6b0d404a1 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:33.500254 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-62ec05da-d27f-4ef3-8fba-ff826b82cbcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:33.520063 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1258/2479] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:40:33 2022] PUT /identity/v3/projects/7a9fe07c0f9b4a71b8dbf3a51919c9b8/users/d2b37d4e1c4c4b7a9b7791e091d4e3cf/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:40:33.526812 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266fdbc0-4135-49cf-8a1f-13f6b0d404a1 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:33.533046 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:40:33.631213 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec05da-d27f-4ef3-8fba-ff826b82cbcb admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:33.636233 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266fdbc0-4135-49cf-8a1f-13f6b0d404a1 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:33.641397 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953ab7ad-f8eb-4667-93e6-2f931e5b406e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:33.642130 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ab7ad-f8eb-4667-93e6-2f931e5b406e admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:33.642447 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ab7ad-f8eb-4667-93e6-2f931e5b406e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:33.642722 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ab7ad-f8eb-4667-93e6-2f931e5b406e admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:33.643454 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953ab7ad-f8eb-4667-93e6-2f931e5b406e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:33.645062 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953ab7ad-f8eb-4667-93e6-2f931e5b406e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:33.656512 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1259/2480] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 21:40:33.660695 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953ab7ad-f8eb-4667-93e6-2f931e5b406e admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:33.672679 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953ab7ad-f8eb-4667-93e6-2f931e5b406e admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:33.678772 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc58f190-4a40-443a-9c84-e0d87387b159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:33.680519 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58f190-4a40-443a-9c84-e0d87387b159 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:33.680871 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58f190-4a40-443a-9c84-e0d87387b159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:33.681216 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58f190-4a40-443a-9c84-e0d87387b159 admin admin] PATH_INFO: `/v3/projects/7a9fe07c0f9b4a71b8dbf3a51919c9b8/users/d2b37d4e1c4c4b7a9b7791e091d4e3cf/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:33.683476 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc58f190-4a40-443a-9c84-e0d87387b159 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a9fe07c0f9b4a71b8dbf3a51919c9b8, user_id=d2b37d4e1c4c4b7a9b7791e091d4e3cf, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:33.687833 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc58f190-4a40-443a-9c84-e0d87387b159 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:33.688912 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:40:33.701402 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1260/2481] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:40:33 2022] PUT /identity/v3/projects/7a9fe07c0f9b4a71b8dbf3a51919c9b8/users/d2b37d4e1c4c4b7a9b7791e091d4e3cf/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:40:33.713014 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843514c4-a453-4789-8c46-11692cabdf36 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:33.713506 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843514c4-a453-4789-8c46-11692cabdf36 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:33.713733 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef77dd79-3054-4ed2-8b61-ca1ec57eadee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6143bddc05a46ea972709576c7ad95f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:33.713890 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843514c4-a453-4789-8c46-11692cabdf36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:33.714456 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef77dd79-3054-4ed2-8b61-ca1ec57eadee service placement] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:33.714744 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef77dd79-3054-4ed2-8b61-ca1ec57eadee service placement] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:33.715455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef77dd79-3054-4ed2-8b61-ca1ec57eadee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:33.717052 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ef77dd79-3054-4ed2-8b61-ca1ec57eadee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:33.732773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef77dd79-3054-4ed2-8b61-ca1ec57eadee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:33.734323 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef77dd79-3054-4ed2-8b61-ca1ec57eadee service placement] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:33.748219 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1222/2482] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:40:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:40:33.754670 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-843514c4-a453-4789-8c46-11692cabdf36 None None] MFA Rules not processed for user `d2b37d4e1c4c4b7a9b7791e091d4e3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:40:33.756510 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-843514c4-a453-4789-8c46-11692cabdf36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:33.862469 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1261/2483] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:40:33 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:40:33.881285 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef77dd79-3054-4ed2-8b61-ca1ec57eadee service placement] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:33.884111 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ef77dd79-3054-4ed2-8b61-ca1ec57eadee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:34.015987 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef77dd79-3054-4ed2-8b61-ca1ec57eadee service placement] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:34.073552 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc30f0f9-9fd4-4651-8ab4-adfb50316930 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:34.074501 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30f0f9-9fd4-4651-8ab4-adfb50316930 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:34.074501 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30f0f9-9fd4-4651-8ab4-adfb50316930 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:34.074828 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30f0f9-9fd4-4651-8ab4-adfb50316930 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:34.087172 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc30f0f9-9fd4-4651-8ab4-adfb50316930 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:34.088670 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc30f0f9-9fd4-4651-8ab4-adfb50316930 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:34.099781 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1223/2484] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:40:33 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:40:34.236825 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843514c4-a453-4789-8c46-11692cabdf36 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:34.303922 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843514c4-a453-4789-8c46-11692cabdf36 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:34.308541 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a97bb6-733b-4ba1-a05a-fa94f6290924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:34.309436 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a97bb6-733b-4ba1-a05a-fa94f6290924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:34.309718 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a97bb6-733b-4ba1-a05a-fa94f6290924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:34.309989 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a97bb6-733b-4ba1-a05a-fa94f6290924 admin admin] PATH_INFO: `/v3/users/5a274fb66ff64d2e88232c2b6ea3e013` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:34.311478 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a97bb6-733b-4ba1-a05a-fa94f6290924 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a274fb66ff64d2e88232c2b6ea3e013)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:34.313491 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a97bb6-733b-4ba1-a05a-fa94f6290924 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:34.327515 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:40:34.367297 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-14a97bb6-733b-4ba1-a05a-fa94f6290924 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a274fb66ff64d2e88232c2b6ea3e013'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:34.382287 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-14a97bb6-733b-4ba1-a05a-fa94f6290924 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a274fb66ff64d2e88232c2b6ea3e013'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:34.383030 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-14a97bb6-733b-4ba1-a05a-fa94f6290924 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a274fb66ff64d2e88232c2b6ea3e013'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:34.383724 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-14a97bb6-733b-4ba1-a05a-fa94f6290924 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a274fb66ff64d2e88232c2b6ea3e013'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:34.391628 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-14a97bb6-733b-4ba1-a05a-fa94f6290924 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a274fb66ff64d2e88232c2b6ea3e013'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:34.406030 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1262/2485] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:40:34 2022] DELETE /identity/v3/users/5a274fb66ff64d2e88232c2b6ea3e013 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:40:34.410632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b67a98-ba8d-4b45-b568-7080c9627f36 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:34.411164 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b67a98-ba8d-4b45-b568-7080c9627f36 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:34.411317 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b67a98-ba8d-4b45-b568-7080c9627f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:34.429488 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-53b67a98-ba8d-4b45-b568-7080c9627f36 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:40:34.430878 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-53b67a98-ba8d-4b45-b568-7080c9627f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:34.496237 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1224/2486] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:40:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:40:34.506437 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a97bb6-733b-4ba1-a05a-fa94f6290924 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:34.508080 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-14a97bb6-733b-4ba1-a05a-fa94f6290924 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:34.566664 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a97bb6-733b-4ba1-a05a-fa94f6290924 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:34.570778 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb0e69-3b0b-4f82-af53-778315babc48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:34.571433 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb0e69-3b0b-4f82-af53-778315babc48 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:34.571709 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb0e69-3b0b-4f82-af53-778315babc48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:34.571973 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb0e69-3b0b-4f82-af53-778315babc48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:34.582463 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cb0e69-3b0b-4f82-af53-778315babc48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:34.583931 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cb0e69-3b0b-4f82-af53-778315babc48 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:34.600783 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1263/2487] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:40:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:40:34.911045 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b67a98-ba8d-4b45-b568-7080c9627f36 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:34.912587 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-53b67a98-ba8d-4b45-b568-7080c9627f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:34.926971 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b67a98-ba8d-4b45-b568-7080c9627f36 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:34.942505 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed0c9e-1e37-494c-b285-2a2afac864f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:34.943282 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed0c9e-1e37-494c-b285-2a2afac864f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:34.943574 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed0c9e-1e37-494c-b285-2a2afac864f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:34.943863 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed0c9e-1e37-494c-b285-2a2afac864f1 admin admin] PATH_INFO: `/v3/projects/14328cf4dee4416e83699f945c365049` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:34.945256 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ed0c9e-1e37-494c-b285-2a2afac864f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14328cf4dee4416e83699f945c365049)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:34.947270 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ed0c9e-1e37-494c-b285-2a2afac864f1 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:34.986393 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:40:35.007708 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-99ed0c9e-1e37-494c-b285-2a2afac864f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14328cf4dee4416e83699f945c365049', 'request_id': 'req-99ed0c9e-1e37-494c-b285-2a2afac864f1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:35.015748 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-99ed0c9e-1e37-494c-b285-2a2afac864f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14328cf4dee4416e83699f945c365049', 'request_id': 'req-99ed0c9e-1e37-494c-b285-2a2afac864f1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:35.026483 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-99ed0c9e-1e37-494c-b285-2a2afac864f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14328cf4dee4416e83699f945c365049', 'request_id': 'req-99ed0c9e-1e37-494c-b285-2a2afac864f1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:35.042725 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-99ed0c9e-1e37-494c-b285-2a2afac864f1 admin admin] The token cache is being invalidate because project 14328cf4dee4416e83699f945c365049 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:40:35.043005 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-99ed0c9e-1e37-494c-b285-2a2afac864f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:35.043783 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-99ed0c9e-1e37-494c-b285-2a2afac864f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:35.045775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1225/2488] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:40:34 2022] DELETE /identity/v3/projects/14328cf4dee4416e83699f945c365049 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:40:35.050064 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb0e69-3b0b-4f82-af53-778315babc48 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:35.060861 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb0e69-3b0b-4f82-af53-778315babc48 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:35.135524 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeb27a9-c1e8-4e1e-9c2e-ac84e4e03f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:35.136625 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb27a9-c1e8-4e1e-9c2e-ac84e4e03f9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:35.137022 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb27a9-c1e8-4e1e-9c2e-ac84e4e03f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:35.137534 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb27a9-c1e8-4e1e-9c2e-ac84e4e03f9c admin admin] PATH_INFO: `/v3/users/d2b37d4e1c4c4b7a9b7791e091d4e3cf` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:35.139460 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeb27a9-c1e8-4e1e-9c2e-ac84e4e03f9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2b37d4e1c4c4b7a9b7791e091d4e3cf)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:35.142710 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeb27a9-c1e8-4e1e-9c2e-ac84e4e03f9c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:35.206075 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:40:35.225228 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1aeb27a9-c1e8-4e1e-9c2e-ac84e4e03f9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2b37d4e1c4c4b7a9b7791e091d4e3cf'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:35.245010 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1aeb27a9-c1e8-4e1e-9c2e-ac84e4e03f9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2b37d4e1c4c4b7a9b7791e091d4e3cf'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:35.245859 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1aeb27a9-c1e8-4e1e-9c2e-ac84e4e03f9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2b37d4e1c4c4b7a9b7791e091d4e3cf'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:35.246576 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1aeb27a9-c1e8-4e1e-9c2e-ac84e4e03f9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2b37d4e1c4c4b7a9b7791e091d4e3cf'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:35.256410 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1aeb27a9-c1e8-4e1e-9c2e-ac84e4e03f9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2b37d4e1c4c4b7a9b7791e091d4e3cf'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:35.273083 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1264/2489] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:40:35 2022] DELETE /identity/v3/users/d2b37d4e1c4c4b7a9b7791e091d4e3cf => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:40:35.570656 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed0c9e-1e37-494c-b285-2a2afac864f1 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:35.573031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-99ed0c9e-1e37-494c-b285-2a2afac864f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:35.591066 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed0c9e-1e37-494c-b285-2a2afac864f1 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:35.658324 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a548e-1dfc-4560-b639-614bdc9b6541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:35.659431 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a548e-1dfc-4560-b639-614bdc9b6541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:35.659794 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a548e-1dfc-4560-b639-614bdc9b6541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:35.660043 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a548e-1dfc-4560-b639-614bdc9b6541 admin admin] PATH_INFO: `/v3/projects/7a9fe07c0f9b4a71b8dbf3a51919c9b8` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:35.661476 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0a548e-1dfc-4560-b639-614bdc9b6541 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a9fe07c0f9b4a71b8dbf3a51919c9b8)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:35.663554 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0a548e-1dfc-4560-b639-614bdc9b6541 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:35.717294 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9a0a548e-1dfc-4560-b639-614bdc9b6541 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a9fe07c0f9b4a71b8dbf3a51919c9b8', 'request_id': 'req-9a0a548e-1dfc-4560-b639-614bdc9b6541'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:35.724221 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9a0a548e-1dfc-4560-b639-614bdc9b6541 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a9fe07c0f9b4a71b8dbf3a51919c9b8', 'request_id': 'req-9a0a548e-1dfc-4560-b639-614bdc9b6541'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:35.740605 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9a0a548e-1dfc-4560-b639-614bdc9b6541 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a9fe07c0f9b4a71b8dbf3a51919c9b8', 'request_id': 'req-9a0a548e-1dfc-4560-b639-614bdc9b6541'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:35.753600 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9a0a548e-1dfc-4560-b639-614bdc9b6541 admin admin] The token cache is being invalidate because project 7a9fe07c0f9b4a71b8dbf3a51919c9b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:40:35.753810 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9a0a548e-1dfc-4560-b639-614bdc9b6541 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:35.754332 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9a0a548e-1dfc-4560-b639-614bdc9b6541 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:35.756136 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1226/2490] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:40:35 2022] DELETE /identity/v3/projects/7a9fe07c0f9b4a71b8dbf3a51919c9b8 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:40:35.762949 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abfcbf3-5a6b-4446-8d17-2f781163b46e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:35.763703 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abfcbf3-5a6b-4446-8d17-2f781163b46e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:35.763703 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abfcbf3-5a6b-4446-8d17-2f781163b46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:35.781418 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-1abfcbf3-5a6b-4446-8d17-2f781163b46e None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:40:35.782683 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1abfcbf3-5a6b-4446-8d17-2f781163b46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:35.868570 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1265/2491] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:40:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:40:35.877602 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f527d71-d746-4933-a3fe-aafbe8db93c0 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:35.877900 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f527d71-d746-4933-a3fe-aafbe8db93c0 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:35.878175 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f527d71-d746-4933-a3fe-aafbe8db93c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:35.897839 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-4f527d71-d746-4933-a3fe-aafbe8db93c0 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:40:35.899085 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4f527d71-d746-4933-a3fe-aafbe8db93c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:35.911241 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1227/2492] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:40:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:40:35.915277 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abfcbf3-5a6b-4446-8d17-2f781163b46e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:35.936690 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abfcbf3-5a6b-4446-8d17-2f781163b46e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:35.943523 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6c73b2-1906-45a5-a099-143070a0a998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:35.944211 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6c73b2-1906-45a5-a099-143070a0a998 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:35.944553 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6c73b2-1906-45a5-a099-143070a0a998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:35.945323 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6c73b2-1906-45a5-a099-143070a0a998 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:35.946068 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6c73b2-1906-45a5-a099-143070a0a998 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:40:35.946403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6c73b2-1906-45a5-a099-143070a0a998 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:35.948056 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6c73b2-1906-45a5-a099-143070a0a998 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:35.964992 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1266/2493] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:40:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:40:35.966596 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f527d71-d746-4933-a3fe-aafbe8db93c0 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:35.978955 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f527d71-d746-4933-a3fe-aafbe8db93c0 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:35.984545 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3805ecae-2e71-4d69-8885-1e017d342d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:35.985677 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3805ecae-2e71-4d69-8885-1e017d342d95 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:35.985677 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3805ecae-2e71-4d69-8885-1e017d342d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:35.985879 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3805ecae-2e71-4d69-8885-1e017d342d95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:35.986892 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3805ecae-2e71-4d69-8885-1e017d342d95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:35.988540 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3805ecae-2e71-4d69-8885-1e017d342d95 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:36.024829 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1228/2494] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:40:35 2022] POST /identity/v3/projects => generated 371 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:40:36.029627 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6c73b2-1906-45a5-a099-143070a0a998 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:36.043442 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6c73b2-1906-45a5-a099-143070a0a998 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:36.131315 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b715e97-384a-4932-8982-8636e35e93ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:36.131559 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b715e97-384a-4932-8982-8636e35e93ab admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:36.131801 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b715e97-384a-4932-8982-8636e35e93ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:36.132060 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b715e97-384a-4932-8982-8636e35e93ab admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:36.133405 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b715e97-384a-4932-8982-8636e35e93ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:36.134928 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b715e97-384a-4932-8982-8636e35e93ab admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:36.169307 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1267/2495] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:40:36 2022] POST /identity/v3/users => generated 286 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:40:36.174267 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3805ecae-2e71-4d69-8885-1e017d342d95 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:36.185719 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3805ecae-2e71-4d69-8885-1e017d342d95 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:36.199764 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9420a47-3dbc-41be-b4ad-8fa8ff81d189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:36.201358 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9420a47-3dbc-41be-b4ad-8fa8ff81d189 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:36.201612 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9420a47-3dbc-41be-b4ad-8fa8ff81d189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:36.202493 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9420a47-3dbc-41be-b4ad-8fa8ff81d189 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:36.203744 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9420a47-3dbc-41be-b4ad-8fa8ff81d189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:36.206023 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9420a47-3dbc-41be-b4ad-8fa8ff81d189 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:36.221532 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1229/2496] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:40:36 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 21:40:36.225558 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b715e97-384a-4932-8982-8636e35e93ab admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:36.236042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b715e97-384a-4932-8982-8636e35e93ab admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:36.241126 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5012eba3-48a6-4c22-a52b-523bdd4811cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:36.241911 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012eba3-48a6-4c22-a52b-523bdd4811cf admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:36.242289 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012eba3-48a6-4c22-a52b-523bdd4811cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:36.242642 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012eba3-48a6-4c22-a52b-523bdd4811cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:36.243512 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5012eba3-48a6-4c22-a52b-523bdd4811cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:36.245232 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5012eba3-48a6-4c22-a52b-523bdd4811cf admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:36.259805 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1268/2497] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:40:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:40:36.261347 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9420a47-3dbc-41be-b4ad-8fa8ff81d189 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:36.271856 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9420a47-3dbc-41be-b4ad-8fa8ff81d189 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:36.282034 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c7d31-239b-4c24-9910-e1acdca2bc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:36.283517 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c7d31-239b-4c24-9910-e1acdca2bc96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:36.284051 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c7d31-239b-4c24-9910-e1acdca2bc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:36.284397 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c7d31-239b-4c24-9910-e1acdca2bc96 admin admin] PATH_INFO: `/v3/projects/0270eeeae10141099581f1ad2e9534fd/users/2007a4ed35d441ed90df96746c7d4e0a/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:36.287021 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:40:36.306763 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c7d31-239b-4c24-9910-e1acdca2bc96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0270eeeae10141099581f1ad2e9534fd, user_id=2007a4ed35d441ed90df96746c7d4e0a, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:36.311419 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c7d31-239b-4c24-9910-e1acdca2bc96 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:36.343846 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1230/2498] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:40:36 2022] PUT /identity/v3/projects/0270eeeae10141099581f1ad2e9534fd/users/2007a4ed35d441ed90df96746c7d4e0a/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:40:36.344769 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09681c-5993-4bab-a063-543cfb12b59a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:36.345061 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09681c-5993-4bab-a063-543cfb12b59a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:36.345341 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09681c-5993-4bab-a063-543cfb12b59a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:36.384709 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-5d09681c-5993-4bab-a063-543cfb12b59a None None] MFA Rules not processed for user `2007a4ed35d441ed90df96746c7d4e0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:40:36.386221 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5d09681c-5993-4bab-a063-543cfb12b59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:36.393225 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:40:36.500371 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1269/2499] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:40:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:40:36.513890 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c7d31-239b-4c24-9910-e1acdca2bc96 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:36.519436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b98c7d31-239b-4c24-9910-e1acdca2bc96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:36.600645 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c7d31-239b-4c24-9910-e1acdca2bc96 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:36.604966 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a60aec-f5ce-4a25-b977-45ed79a21cc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:36.606049 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a60aec-f5ce-4a25-b977-45ed79a21cc8 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:36.607105 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a60aec-f5ce-4a25-b977-45ed79a21cc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:36.607541 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a60aec-f5ce-4a25-b977-45ed79a21cc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:36.618946 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a60aec-f5ce-4a25-b977-45ed79a21cc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:36.621301 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a60aec-f5ce-4a25-b977-45ed79a21cc8 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:36.633671 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1231/2500] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:40:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:40:36.657600 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d09681c-5993-4bab-a063-543cfb12b59a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:36.736404 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d09681c-5993-4bab-a063-543cfb12b59a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:36.741844 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe0aae-56ba-4fda-82cf-c3b745763610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:36.742921 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe0aae-56ba-4fda-82cf-c3b745763610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:36.743315 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe0aae-56ba-4fda-82cf-c3b745763610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:36.743697 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe0aae-56ba-4fda-82cf-c3b745763610 admin admin] PATH_INFO: `/v3/users/2007a4ed35d441ed90df96746c7d4e0a` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:36.745428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afe0aae-56ba-4fda-82cf-c3b745763610 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2007a4ed35d441ed90df96746c7d4e0a)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:36.747079 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afe0aae-56ba-4fda-82cf-c3b745763610 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:36.820444 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3afe0aae-56ba-4fda-82cf-c3b745763610 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2007a4ed35d441ed90df96746c7d4e0a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:36.850607 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3afe0aae-56ba-4fda-82cf-c3b745763610 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2007a4ed35d441ed90df96746c7d4e0a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:36.851745 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3afe0aae-56ba-4fda-82cf-c3b745763610 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2007a4ed35d441ed90df96746c7d4e0a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:36.852567 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3afe0aae-56ba-4fda-82cf-c3b745763610 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2007a4ed35d441ed90df96746c7d4e0a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:36.865893 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3afe0aae-56ba-4fda-82cf-c3b745763610 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2007a4ed35d441ed90df96746c7d4e0a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:36.894418 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1270/2501] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:40:36 2022] DELETE /identity/v3/users/2007a4ed35d441ed90df96746c7d4e0a => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:40:36.900205 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936829a-0eab-49e9-8bf2-cb194c25b73e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:36.900510 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936829a-0eab-49e9-8bf2-cb194c25b73e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:36.900813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936829a-0eab-49e9-8bf2-cb194c25b73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:36.922634 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-1936829a-0eab-49e9-8bf2-cb194c25b73e None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:40:36.923863 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1936829a-0eab-49e9-8bf2-cb194c25b73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:37.001442 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1232/2502] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:40:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:40:37.009630 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe0aae-56ba-4fda-82cf-c3b745763610 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:37.011277 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3afe0aae-56ba-4fda-82cf-c3b745763610 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:37.054763 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:40:37.098404 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe0aae-56ba-4fda-82cf-c3b745763610 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:37.104735 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21eb702-8d42-4b23-ade7-25aef688c764 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:37.105808 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21eb702-8d42-4b23-ade7-25aef688c764 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:37.106320 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21eb702-8d42-4b23-ade7-25aef688c764 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:37.106749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21eb702-8d42-4b23-ade7-25aef688c764 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:37.119990 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21eb702-8d42-4b23-ade7-25aef688c764 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:37.121908 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21eb702-8d42-4b23-ade7-25aef688c764 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:37.149932 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1271/2503] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:40:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:40:37.458914 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1936829a-0eab-49e9-8bf2-cb194c25b73e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:37.460474 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1936829a-0eab-49e9-8bf2-cb194c25b73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:37.473726 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1936829a-0eab-49e9-8bf2-cb194c25b73e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:37.479252 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986e23e5-64e4-417d-8d53-60ed5f991312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:37.480158 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e23e5-64e4-417d-8d53-60ed5f991312 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:37.480439 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e23e5-64e4-417d-8d53-60ed5f991312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:37.480712 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e23e5-64e4-417d-8d53-60ed5f991312 admin admin] PATH_INFO: `/v3/projects/0270eeeae10141099581f1ad2e9534fd` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:37.482125 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986e23e5-64e4-417d-8d53-60ed5f991312 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0270eeeae10141099581f1ad2e9534fd)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:37.484298 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986e23e5-64e4-417d-8d53-60ed5f991312 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:37.540311 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-986e23e5-64e4-417d-8d53-60ed5f991312 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0270eeeae10141099581f1ad2e9534fd', 'request_id': 'req-986e23e5-64e4-417d-8d53-60ed5f991312'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:37.547217 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-986e23e5-64e4-417d-8d53-60ed5f991312 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0270eeeae10141099581f1ad2e9534fd', 'request_id': 'req-986e23e5-64e4-417d-8d53-60ed5f991312'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:37.564516 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-986e23e5-64e4-417d-8d53-60ed5f991312 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0270eeeae10141099581f1ad2e9534fd', 'request_id': 'req-986e23e5-64e4-417d-8d53-60ed5f991312'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:37.579622 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-986e23e5-64e4-417d-8d53-60ed5f991312 admin admin] The token cache is being invalidate because project 0270eeeae10141099581f1ad2e9534fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:40:37.579954 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-986e23e5-64e4-417d-8d53-60ed5f991312 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:37.580697 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-986e23e5-64e4-417d-8d53-60ed5f991312 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:37.586073 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1233/2504] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:40:37 2022] DELETE /identity/v3/projects/0270eeeae10141099581f1ad2e9534fd => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:40:37.589906 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b18cf7-5799-454f-a562-61445afab23b None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:37.590460 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b18cf7-5799-454f-a562-61445afab23b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:37.590967 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b18cf7-5799-454f-a562-61445afab23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:37.609760 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e0b18cf7-5799-454f-a562-61445afab23b None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:40:37.610931 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e0b18cf7-5799-454f-a562-61445afab23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:37.678982 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1272/2505] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:40:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:40:37.688858 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31dae90-c1e5-4ccd-b7d0-5e5a31c2991c None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:37.689517 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31dae90-c1e5-4ccd-b7d0-5e5a31c2991c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:37.689517 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31dae90-c1e5-4ccd-b7d0-5e5a31c2991c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:37.709474 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b31dae90-c1e5-4ccd-b7d0-5e5a31c2991c None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:40:37.710918 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b31dae90-c1e5-4ccd-b7d0-5e5a31c2991c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:37.719410 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1234/2506] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:40:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:40:37.724635 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b18cf7-5799-454f-a562-61445afab23b None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:37.735227 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b18cf7-5799-454f-a562-61445afab23b None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:37.740948 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4bb05a-826d-4dab-b43d-fbce5744202a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:37.741850 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4bb05a-826d-4dab-b43d-fbce5744202a admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:37.742071 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4bb05a-826d-4dab-b43d-fbce5744202a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:37.742195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4bb05a-826d-4dab-b43d-fbce5744202a admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:37.743091 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4bb05a-826d-4dab-b43d-fbce5744202a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:40:37.743311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4bb05a-826d-4dab-b43d-fbce5744202a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:37.744971 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4bb05a-826d-4dab-b43d-fbce5744202a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:37.760111 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1273/2507] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:40:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:40:37.765394 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31dae90-c1e5-4ccd-b7d0-5e5a31c2991c None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:37.770013 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:40:37.777490 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31dae90-c1e5-4ccd-b7d0-5e5a31c2991c None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:37.783860 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fffcd7a-9fbd-4ab5-99d0-7db0344eb150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:37.784681 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fffcd7a-9fbd-4ab5-99d0-7db0344eb150 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:37.784953 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fffcd7a-9fbd-4ab5-99d0-7db0344eb150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:37.785275 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fffcd7a-9fbd-4ab5-99d0-7db0344eb150 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:37.785964 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fffcd7a-9fbd-4ab5-99d0-7db0344eb150 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:37.787917 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fffcd7a-9fbd-4ab5-99d0-7db0344eb150 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:37.815687 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1235/2508] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:40:37 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:40:37.819827 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4bb05a-826d-4dab-b43d-fbce5744202a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:37.830822 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4bb05a-826d-4dab-b43d-fbce5744202a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:37.898848 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6987b7b-ae02-4ffa-b9b7-42de18c235fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:37.899088 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6987b7b-ae02-4ffa-b9b7-42de18c235fd admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:37.899490 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6987b7b-ae02-4ffa-b9b7-42de18c235fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:37.899838 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6987b7b-ae02-4ffa-b9b7-42de18c235fd admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:37.900695 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6987b7b-ae02-4ffa-b9b7-42de18c235fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:37.903342 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6987b7b-ae02-4ffa-b9b7-42de18c235fd admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:37.932990 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1274/2509] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:40:37 2022] POST /identity/v3/users => generated 291 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:40:37.933441 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fffcd7a-9fbd-4ab5-99d0-7db0344eb150 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:37.945210 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fffcd7a-9fbd-4ab5-99d0-7db0344eb150 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:37.950582 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b421b4f5-ec70-4f84-a530-6e236d528824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:37.951332 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b421b4f5-ec70-4f84-a530-6e236d528824 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:37.951611 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b421b4f5-ec70-4f84-a530-6e236d528824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:37.952069 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b421b4f5-ec70-4f84-a530-6e236d528824 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:37.953212 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b421b4f5-ec70-4f84-a530-6e236d528824 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:37.954812 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b421b4f5-ec70-4f84-a530-6e236d528824 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:37.969739 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1236/2510] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 21:40:37.973813 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6987b7b-ae02-4ffa-b9b7-42de18c235fd admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:37.988584 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6987b7b-ae02-4ffa-b9b7-42de18c235fd admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:37.993934 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f568284-523b-4b0a-9256-6f4329f1261c None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:37.994108 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17176e-1b4c-4c2f-a26c-66779f3e7869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:37.994540 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f568284-523b-4b0a-9256-6f4329f1261c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:37.994751 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17176e-1b4c-4c2f-a26c-66779f3e7869 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:37.994953 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17176e-1b4c-4c2f-a26c-66779f3e7869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:37.995072 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f568284-523b-4b0a-9256-6f4329f1261c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:37.995232 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17176e-1b4c-4c2f-a26c-66779f3e7869 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:37.995971 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c17176e-1b4c-4c2f-a26c-66779f3e7869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:37.997859 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c17176e-1b4c-4c2f-a26c-66779f3e7869 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:38.011733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1275/2511] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:40:38.015267 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17176e-1b4c-4c2f-a26c-66779f3e7869 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:38.016431 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-6f568284-523b-4b0a-9256-6f4329f1261c None None] MFA Rules not processed for user `eba91dff7cac401e81f6fb6391874af1`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:40:38.018021 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6f568284-523b-4b0a-9256-6f4329f1261c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:38.026329 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17176e-1b4c-4c2f-a26c-66779f3e7869 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:38.035065 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebcc948-a00d-4e3b-a650-2da37fc39016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:38.036241 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebcc948-a00d-4e3b-a650-2da37fc39016 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:38.036629 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebcc948-a00d-4e3b-a650-2da37fc39016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:38.037151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebcc948-a00d-4e3b-a650-2da37fc39016 admin admin] PATH_INFO: `/v3/projects/336de274dc52432f879646a6d0a42612/users/52c2372de2624b5c9df4ed3d5caf7f3f/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:38.057938 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebcc948-a00d-4e3b-a650-2da37fc39016 admin admin] RBAC: Authorizing `identity:create_grant(project_id=336de274dc52432f879646a6d0a42612, user_id=52c2372de2624b5c9df4ed3d5caf7f3f, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:38.061559 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebcc948-a00d-4e3b-a650-2da37fc39016 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:38.079550 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1276/2512] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:40:38 2022] PUT /identity/v3/projects/336de274dc52432f879646a6d0a42612/users/52c2372de2624b5c9df4ed3d5caf7f3f/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:40:38.087010 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1237/2513] 10.222.0.65 () {58 vars in 967 bytes} [Thu May 19 21:40:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:40:38.091414 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27240ee0-ccbd-4afc-9bdc-32302d686e4f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:38.091720 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27240ee0-ccbd-4afc-9bdc-32302d686e4f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:38.092002 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27240ee0-ccbd-4afc-9bdc-32302d686e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:38.116069 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:40:38.129526 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-27240ee0-ccbd-4afc-9bdc-32302d686e4f None None] MFA Rules not processed for user `52c2372de2624b5c9df4ed3d5caf7f3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:40:38.131422 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-27240ee0-ccbd-4afc-9bdc-32302d686e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:38.143952 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f568284-523b-4b0a-9256-6f4329f1261c None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:38.145745 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6f568284-523b-4b0a-9256-6f4329f1261c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:38.224492 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1277/2514] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:40:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:40:38.235251 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f568284-523b-4b0a-9256-6f4329f1261c None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:38.237183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27240ee0-ccbd-4afc-9bdc-32302d686e4f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:38.239334 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-27240ee0-ccbd-4afc-9bdc-32302d686e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:38.306623 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27240ee0-ccbd-4afc-9bdc-32302d686e4f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:38.310605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b6b17-dd72-4cf7-9139-4528a0a9277d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:38.312245 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b6b17-dd72-4cf7-9139-4528a0a9277d service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:38.312245 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b6b17-dd72-4cf7-9139-4528a0a9277d service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:38.312534 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b6b17-dd72-4cf7-9139-4528a0a9277d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:38.319929 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b3ac3-d34f-4721-8cd9-e87c8c5d8976 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e07984b36934aff9052541ef2a56f36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:38.320884 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b3ac3-d34f-4721-8cd9-e87c8c5d8976 service swift] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:38.321168 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b3ac3-d34f-4721-8cd9-e87c8c5d8976 service swift] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:38.321448 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b3ac3-d34f-4721-8cd9-e87c8c5d8976 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:38.326175 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6b6b17-dd72-4cf7-9139-4528a0a9277d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:38.328760 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6b6b17-dd72-4cf7-9139-4528a0a9277d service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:38.344364 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1278/2515] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:40:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:40:38.345706 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:40:38.380174 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b149b0-9cc8-4021-8a37-f22bb27dcaa9 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:38.380430 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b149b0-9cc8-4021-8a37-f22bb27dcaa9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:38.382990 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b149b0-9cc8-4021-8a37-f22bb27dcaa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:38.408587 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864b3ac3-d34f-4721-8cd9-e87c8c5d8976 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:38.410208 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864b3ac3-d34f-4721-8cd9-e87c8c5d8976 service swift] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:38.411007 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-52b149b0-9cc8-4021-8a37-f22bb27dcaa9 None None] MFA Rules not processed for user `6d16feb169ac46f7983a6b8045de77c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:40:38.412215 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-52b149b0-9cc8-4021-8a37-f22bb27dcaa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:38.422404 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1238/2516] 10.222.0.65 () {60 vars in 1384 bytes} [Thu May 19 21:40:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 19 21:40:38.481185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1279/2517] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:40:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:40:38.856494 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b3ac3-d34f-4721-8cd9-e87c8c5d8976 service swift] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:38.859384 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-864b3ac3-d34f-4721-8cd9-e87c8c5d8976 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:38.928151 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b3ac3-d34f-4721-8cd9-e87c8c5d8976 service swift] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:38.932263 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0867557-09b8-4eab-ad46-1bbe801a1a25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:38.933075 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0867557-09b8-4eab-ad46-1bbe801a1a25 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:38.933362 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0867557-09b8-4eab-ad46-1bbe801a1a25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:38.933644 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0867557-09b8-4eab-ad46-1bbe801a1a25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:38.935550 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b0867557-09b8-4eab-ad46-1bbe801a1a25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:39.003348 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0867557-09b8-4eab-ad46-1bbe801a1a25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:39.005759 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0867557-09b8-4eab-ad46-1bbe801a1a25 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:39.023842 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1239/2518] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:40:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:40:46.913435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b149b0-9cc8-4021-8a37-f22bb27dcaa9 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:46.915938 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-52b149b0-9cc8-4021-8a37-f22bb27dcaa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:46.999554 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0867557-09b8-4eab-ad46-1bbe801a1a25 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:47.002283 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:40:47.003848 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b0867557-09b8-4eab-ad46-1bbe801a1a25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:47.029629 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0867557-09b8-4eab-ad46-1bbe801a1a25 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:47.033544 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b149b0-9cc8-4021-8a37-f22bb27dcaa9 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:47.037937 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14aff8-15ad-4a3e-bf7c-57432eb4ebdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:47.039493 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14aff8-15ad-4a3e-bf7c-57432eb4ebdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:47.039783 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1316b3c-3cbc-4b48-bd12-660f65b67c10 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6143bddc05a46ea972709576c7ad95f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:47.039997 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14aff8-15ad-4a3e-bf7c-57432eb4ebdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:47.040495 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14aff8-15ad-4a3e-bf7c-57432eb4ebdb admin admin] PATH_INFO: `/v3/users/1664dba0cd9b4d9da1e4b78691b8567e` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:47.040745 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1316b3c-3cbc-4b48-bd12-660f65b67c10 service placement] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:47.041119 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1316b3c-3cbc-4b48-bd12-660f65b67c10 service placement] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:47.041491 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1316b3c-3cbc-4b48-bd12-660f65b67c10 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:47.042969 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c14aff8-15ad-4a3e-bf7c-57432eb4ebdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=1664dba0cd9b4d9da1e4b78691b8567e)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:47.043820 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b1316b3c-3cbc-4b48-bd12-660f65b67c10 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:47.047484 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c14aff8-15ad-4a3e-bf7c-57432eb4ebdb admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:47.062583 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1316b3c-3cbc-4b48-bd12-660f65b67c10 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:47.064510 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1316b3c-3cbc-4b48-bd12-660f65b67c10 service placement] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:47.078795 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1280/2519] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:40:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:40:47.136984 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0c14aff8-15ad-4a3e-bf7c-57432eb4ebdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1664dba0cd9b4d9da1e4b78691b8567e'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:47.138526 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0c14aff8-15ad-4a3e-bf7c-57432eb4ebdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1664dba0cd9b4d9da1e4b78691b8567e'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:47.153125 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:40:47.154838 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0c14aff8-15ad-4a3e-bf7c-57432eb4ebdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1664dba0cd9b4d9da1e4b78691b8567e'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:47.167120 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0c14aff8-15ad-4a3e-bf7c-57432eb4ebdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1664dba0cd9b4d9da1e4b78691b8567e'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:47.178826 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0c14aff8-15ad-4a3e-bf7c-57432eb4ebdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1664dba0cd9b4d9da1e4b78691b8567e'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:47.187269 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1316b3c-3cbc-4b48-bd12-660f65b67c10 service placement] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:47.192668 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1240/2520] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:40:46 2022] DELETE /identity/v3/users/1664dba0cd9b4d9da1e4b78691b8567e => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:40:47.200656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1316b3c-3cbc-4b48-bd12-660f65b67c10 service placement] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:47.269678 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27558251-5830-4e0b-885e-ed6d2cd0c017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:47.270843 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27558251-5830-4e0b-885e-ed6d2cd0c017 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:47.271402 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27558251-5830-4e0b-885e-ed6d2cd0c017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:47.271803 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27558251-5830-4e0b-885e-ed6d2cd0c017 admin admin] PATH_INFO: `/v3/users/52c2372de2624b5c9df4ed3d5caf7f3f` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:47.273249 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27558251-5830-4e0b-885e-ed6d2cd0c017 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52c2372de2624b5c9df4ed3d5caf7f3f)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:47.275438 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27558251-5830-4e0b-885e-ed6d2cd0c017 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:47.341043 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-27558251-5830-4e0b-885e-ed6d2cd0c017 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52c2372de2624b5c9df4ed3d5caf7f3f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:47.355305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-27558251-5830-4e0b-885e-ed6d2cd0c017 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52c2372de2624b5c9df4ed3d5caf7f3f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:47.356202 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-27558251-5830-4e0b-885e-ed6d2cd0c017 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52c2372de2624b5c9df4ed3d5caf7f3f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:47.357195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-27558251-5830-4e0b-885e-ed6d2cd0c017 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52c2372de2624b5c9df4ed3d5caf7f3f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:47.365179 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-27558251-5830-4e0b-885e-ed6d2cd0c017 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52c2372de2624b5c9df4ed3d5caf7f3f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:47.381541 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1281/2521] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:40:47 2022] DELETE /identity/v3/users/52c2372de2624b5c9df4ed3d5caf7f3f => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:40:47.386420 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbe23b-b6ec-4903-98fc-d493ba2860e5 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:47.386762 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbe23b-b6ec-4903-98fc-d493ba2860e5 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:47.387069 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbe23b-b6ec-4903-98fc-d493ba2860e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:47.408767 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-ddfbe23b-b6ec-4903-98fc-d493ba2860e5 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:40:47.409996 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ddfbe23b-b6ec-4903-98fc-d493ba2860e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:47.417359 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27558251-5830-4e0b-885e-ed6d2cd0c017 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:47.419206 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-27558251-5830-4e0b-885e-ed6d2cd0c017 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:47.503327 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1241/2522] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:40:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:40:47.507698 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:40:47.512259 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27558251-5830-4e0b-885e-ed6d2cd0c017 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:47.513921 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfbe23b-b6ec-4903-98fc-d493ba2860e5 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:47.516561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ddfbe23b-b6ec-4903-98fc-d493ba2860e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:47.520025 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11501f-21f6-4fab-b2a4-82b68343b3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:47.521565 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11501f-21f6-4fab-b2a4-82b68343b3a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:47.522384 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11501f-21f6-4fab-b2a4-82b68343b3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:47.523864 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11501f-21f6-4fab-b2a4-82b68343b3a1 admin admin] PATH_INFO: `/v3/projects/3fcf03712c6f4926a2a95dfd0519536a` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:47.525205 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f11501f-21f6-4fab-b2a4-82b68343b3a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fcf03712c6f4926a2a95dfd0519536a)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:47.527204 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f11501f-21f6-4fab-b2a4-82b68343b3a1 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:47.579634 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2f11501f-21f6-4fab-b2a4-82b68343b3a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fcf03712c6f4926a2a95dfd0519536a', 'request_id': 'req-2f11501f-21f6-4fab-b2a4-82b68343b3a1'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:47.580424 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfbe23b-b6ec-4903-98fc-d493ba2860e5 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:47.586786 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2f11501f-21f6-4fab-b2a4-82b68343b3a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fcf03712c6f4926a2a95dfd0519536a', 'request_id': 'req-2f11501f-21f6-4fab-b2a4-82b68343b3a1'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:47.593134 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2f11501f-21f6-4fab-b2a4-82b68343b3a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fcf03712c6f4926a2a95dfd0519536a', 'request_id': 'req-2f11501f-21f6-4fab-b2a4-82b68343b3a1'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:47.613046 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2f11501f-21f6-4fab-b2a4-82b68343b3a1 admin admin] The token cache is being invalidate because project 3fcf03712c6f4926a2a95dfd0519536a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:40:47.613336 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2f11501f-21f6-4fab-b2a4-82b68343b3a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:47.614081 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2f11501f-21f6-4fab-b2a4-82b68343b3a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:47.615828 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1282/2523] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:40:47 2022] DELETE /identity/v3/projects/3fcf03712c6f4926a2a95dfd0519536a => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:40:47.619825 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11501f-21f6-4fab-b2a4-82b68343b3a1 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:47.621507 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2f11501f-21f6-4fab-b2a4-82b68343b3a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:47.643590 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1249b4d6-4d6c-45d0-a65c-6627073e37e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:47.644578 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1249b4d6-4d6c-45d0-a65c-6627073e37e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:47.644855 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1249b4d6-4d6c-45d0-a65c-6627073e37e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:47.645127 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1249b4d6-4d6c-45d0-a65c-6627073e37e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:47.647437 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1249b4d6-4d6c-45d0-a65c-6627073e37e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:47.701957 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11501f-21f6-4fab-b2a4-82b68343b3a1 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:47.706375 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:40:47.708923 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1c755-017c-441a-a6a5-a59e3841a444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:47.711214 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1c755-017c-441a-a6a5-a59e3841a444 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:47.711789 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1c755-017c-441a-a6a5-a59e3841a444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:47.712114 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1c755-017c-441a-a6a5-a59e3841a444 admin admin] PATH_INFO: `/v3/users/56142bbce9994044be921a231a51cae4` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:47.713877 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c1c755-017c-441a-a6a5-a59e3841a444 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56142bbce9994044be921a231a51cae4)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:47.718843 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c1c755-017c-441a-a6a5-a59e3841a444 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:47.736600 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1249b4d6-4d6c-45d0-a65c-6627073e37e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:47.739113 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1249b4d6-4d6c-45d0-a65c-6627073e37e5 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:47.750158 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1242/2524] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:40:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:40:47.786142 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d7c1c755-017c-441a-a6a5-a59e3841a444 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56142bbce9994044be921a231a51cae4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:47.806115 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d7c1c755-017c-441a-a6a5-a59e3841a444 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56142bbce9994044be921a231a51cae4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:47.807683 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d7c1c755-017c-441a-a6a5-a59e3841a444 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56142bbce9994044be921a231a51cae4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:47.809953 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d7c1c755-017c-441a-a6a5-a59e3841a444 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56142bbce9994044be921a231a51cae4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:47.825201 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d7c1c755-017c-441a-a6a5-a59e3841a444 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56142bbce9994044be921a231a51cae4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:47.845016 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1283/2525] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:40:47 2022] DELETE /identity/v3/users/56142bbce9994044be921a231a51cae4 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:40:48.159174 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1249b4d6-4d6c-45d0-a65c-6627073e37e5 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:48.167590 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1c755-017c-441a-a6a5-a59e3841a444 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:48.170984 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d7c1c755-017c-441a-a6a5-a59e3841a444 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:48.177953 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1249b4d6-4d6c-45d0-a65c-6627073e37e5 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:48.207672 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:40:48.260900 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3522579b-4c83-4e4a-b2b8-8ce204aea393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:48.261879 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3522579b-4c83-4e4a-b2b8-8ce204aea393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:48.262214 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3522579b-4c83-4e4a-b2b8-8ce204aea393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:48.262605 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3522579b-4c83-4e4a-b2b8-8ce204aea393 admin admin] PATH_INFO: `/v3/projects/81af60892d324c7ea1eb2b04440584b1` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:48.264094 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3522579b-4c83-4e4a-b2b8-8ce204aea393 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81af60892d324c7ea1eb2b04440584b1)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:48.266081 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3522579b-4c83-4e4a-b2b8-8ce204aea393 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:48.288511 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1c755-017c-441a-a6a5-a59e3841a444 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:48.295787 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40513745-9111-48b5-8de0-26f94e100496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:48.296745 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40513745-9111-48b5-8de0-26f94e100496 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:48.297032 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40513745-9111-48b5-8de0-26f94e100496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:48.297312 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40513745-9111-48b5-8de0-26f94e100496 admin admin] PATH_INFO: `/v3/projects/336de274dc52432f879646a6d0a42612` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:48.298566 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40513745-9111-48b5-8de0-26f94e100496 admin admin] RBAC: Authorizing `identity:delete_project(project_id=336de274dc52432f879646a6d0a42612)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:48.300548 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40513745-9111-48b5-8de0-26f94e100496 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:48.315985 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-3522579b-4c83-4e4a-b2b8-8ce204aea393 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81af60892d324c7ea1eb2b04440584b1', 'request_id': 'req-3522579b-4c83-4e4a-b2b8-8ce204aea393'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:48.323376 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-3522579b-4c83-4e4a-b2b8-8ce204aea393 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81af60892d324c7ea1eb2b04440584b1', 'request_id': 'req-3522579b-4c83-4e4a-b2b8-8ce204aea393'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:48.334333 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-3522579b-4c83-4e4a-b2b8-8ce204aea393 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81af60892d324c7ea1eb2b04440584b1', 'request_id': 'req-3522579b-4c83-4e4a-b2b8-8ce204aea393'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:48.346892 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-3522579b-4c83-4e4a-b2b8-8ce204aea393 admin admin] The token cache is being invalidate because project 81af60892d324c7ea1eb2b04440584b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:40:48.347182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-3522579b-4c83-4e4a-b2b8-8ce204aea393 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:48.347919 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-3522579b-4c83-4e4a-b2b8-8ce204aea393 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:48.349854 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1243/2526] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:40:48 2022] DELETE /identity/v3/projects/81af60892d324c7ea1eb2b04440584b1 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:40:48.362485 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-40513745-9111-48b5-8de0-26f94e100496 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '336de274dc52432f879646a6d0a42612', 'request_id': 'req-40513745-9111-48b5-8de0-26f94e100496'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:48.370146 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-40513745-9111-48b5-8de0-26f94e100496 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '336de274dc52432f879646a6d0a42612', 'request_id': 'req-40513745-9111-48b5-8de0-26f94e100496'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:48.378988 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-40513745-9111-48b5-8de0-26f94e100496 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '336de274dc52432f879646a6d0a42612', 'request_id': 'req-40513745-9111-48b5-8de0-26f94e100496'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:48.398492 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-40513745-9111-48b5-8de0-26f94e100496 admin admin] The token cache is being invalidate because project 336de274dc52432f879646a6d0a42612 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:40:48.398917 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-40513745-9111-48b5-8de0-26f94e100496 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:48.399804 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-40513745-9111-48b5-8de0-26f94e100496 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:40:48.401713 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1284/2527] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:40:48 2022] DELETE /identity/v3/projects/336de274dc52432f879646a6d0a42612 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:40:48.408649 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89530d4-7f7f-4fd9-ae4c-8a718d23767d None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:48.409016 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89530d4-7f7f-4fd9-ae4c-8a718d23767d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:48.409299 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89530d4-7f7f-4fd9-ae4c-8a718d23767d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:48.438810 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-c89530d4-7f7f-4fd9-ae4c-8a718d23767d None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:40:48.440115 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c89530d4-7f7f-4fd9-ae4c-8a718d23767d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:48.516759 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1244/2528] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:40:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:40:48.525791 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e27d5-ef44-4a98-af2c-b6e2ab64dc54 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:48.526081 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e27d5-ef44-4a98-af2c-b6e2ab64dc54 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:48.526418 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e27d5-ef44-4a98-af2c-b6e2ab64dc54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:48.549523 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-285e27d5-ef44-4a98-af2c-b6e2ab64dc54 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:40:48.551891 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-285e27d5-ef44-4a98-af2c-b6e2ab64dc54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:48.562897 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1285/2529] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:40:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:40:48.567872 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89530d4-7f7f-4fd9-ae4c-8a718d23767d None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:48.579287 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89530d4-7f7f-4fd9-ae4c-8a718d23767d None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:48.584840 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d05f4c0-ee78-406e-9c2f-ce724ca6b103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:48.585527 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05f4c0-ee78-406e-9c2f-ce724ca6b103 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:48.585809 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05f4c0-ee78-406e-9c2f-ce724ca6b103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:48.586080 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05f4c0-ee78-406e-9c2f-ce724ca6b103 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:48.586878 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d05f4c0-ee78-406e-9c2f-ce724ca6b103 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:40:48.587182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d05f4c0-ee78-406e-9c2f-ce724ca6b103 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:48.588875 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d05f4c0-ee78-406e-9c2f-ce724ca6b103 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:48.603176 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1245/2530] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:40:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:40:48.608331 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e27d5-ef44-4a98-af2c-b6e2ab64dc54 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:48.618161 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e27d5-ef44-4a98-af2c-b6e2ab64dc54 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:48.623999 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f152a5-54e9-4ed7-b9ef-a3519c20320c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:48.624695 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f152a5-54e9-4ed7-b9ef-a3519c20320c admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:48.624975 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f152a5-54e9-4ed7-b9ef-a3519c20320c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:48.625243 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f152a5-54e9-4ed7-b9ef-a3519c20320c admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:48.625936 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f152a5-54e9-4ed7-b9ef-a3519c20320c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:48.627964 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f152a5-54e9-4ed7-b9ef-a3519c20320c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:48.659586 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1286/2531] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:40:48 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:40:48.665281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d05f4c0-ee78-406e-9c2f-ce724ca6b103 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:48.672139 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:40:48.676574 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d05f4c0-ee78-406e-9c2f-ce724ca6b103 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:48.736489 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6654492d-0d55-463a-9056-94a7101c516c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:48.737179 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654492d-0d55-463a-9056-94a7101c516c admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:48.737460 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654492d-0d55-463a-9056-94a7101c516c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:48.737735 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654492d-0d55-463a-9056-94a7101c516c admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:48.738474 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6654492d-0d55-463a-9056-94a7101c516c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:48.740433 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6654492d-0d55-463a-9056-94a7101c516c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:48.766261 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1246/2532] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:40:48 2022] POST /identity/v3/users => generated 285 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:40:48.770638 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f152a5-54e9-4ed7-b9ef-a3519c20320c admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:48.782009 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f152a5-54e9-4ed7-b9ef-a3519c20320c admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:48.787560 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b66e66-e586-48fc-91d7-a56a513d98eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:48.788230 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b66e66-e586-48fc-91d7-a56a513d98eb admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:48.788513 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b66e66-e586-48fc-91d7-a56a513d98eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:48.788797 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b66e66-e586-48fc-91d7-a56a513d98eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:48.789532 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b66e66-e586-48fc-91d7-a56a513d98eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:48.791166 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b66e66-e586-48fc-91d7-a56a513d98eb admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:48.802587 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1287/2533] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:40:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:40:48.808035 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6654492d-0d55-463a-9056-94a7101c516c admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:48.819461 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6654492d-0d55-463a-9056-94a7101c516c admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:48.826317 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2565cafa-87cc-4581-889c-f080e448e267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:48.827160 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565cafa-87cc-4581-889c-f080e448e267 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:48.827896 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565cafa-87cc-4581-889c-f080e448e267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:48.828275 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565cafa-87cc-4581-889c-f080e448e267 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:48.829315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2565cafa-87cc-4581-889c-f080e448e267 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:48.831905 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2565cafa-87cc-4581-889c-f080e448e267 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:48.848831 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1247/2534] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:40:48 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 21:40:48.853631 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b66e66-e586-48fc-91d7-a56a513d98eb admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:48.865223 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b66e66-e586-48fc-91d7-a56a513d98eb admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:48.870394 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a692ac9-c4cb-43a9-af5a-22f2fd7aebc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:48.871316 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a692ac9-c4cb-43a9-af5a-22f2fd7aebc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:48.871602 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a692ac9-c4cb-43a9-af5a-22f2fd7aebc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:48.871887 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a692ac9-c4cb-43a9-af5a-22f2fd7aebc7 admin admin] PATH_INFO: `/v3/projects/62e59f8182c84c899ca2645d002625da/users/14681b91d11746e6951d47e5d31e4d0a/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:48.907825 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a692ac9-c4cb-43a9-af5a-22f2fd7aebc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62e59f8182c84c899ca2645d002625da, user_id=14681b91d11746e6951d47e5d31e4d0a, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:48.910571 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a692ac9-c4cb-43a9-af5a-22f2fd7aebc7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:48.926389 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1288/2535] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:40:48 2022] PUT /identity/v3/projects/62e59f8182c84c899ca2645d002625da/users/14681b91d11746e6951d47e5d31e4d0a/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:40:48.938305 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02ed218-0523-427f-8024-ffae772ca87b None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:48.938592 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02ed218-0523-427f-8024-ffae772ca87b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:48.938879 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02ed218-0523-427f-8024-ffae772ca87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:48.973320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-c02ed218-0523-427f-8024-ffae772ca87b None None] MFA Rules not processed for user `14681b91d11746e6951d47e5d31e4d0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:40:48.974616 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c02ed218-0523-427f-8024-ffae772ca87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:49.053744 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1248/2536] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:40:48 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:40:49.066657 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a692ac9-c4cb-43a9-af5a-22f2fd7aebc7 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:40:49.068977 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2a692ac9-c4cb-43a9-af5a-22f2fd7aebc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:40:49.117573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:40:49.146094 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a692ac9-c4cb-43a9-af5a-22f2fd7aebc7 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:40:49.150101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f570d-f131-4584-b70f-13e527336777 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea874ca4088d4e6c83c9d0497323cf6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:40:49.150777 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f570d-f131-4584-b70f-13e527336777 service glance] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:40:49.151060 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f570d-f131-4584-b70f-13e527336777 service glance] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:40:49.151360 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f570d-f131-4584-b70f-13e527336777 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:40:49.161581 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3f570d-f131-4584-b70f-13e527336777 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:40:49.163112 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3f570d-f131-4584-b70f-13e527336777 service glance] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:40:49.174528 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1289/2537] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:40:49 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:40:59.045294 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=64,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:40:59.170505 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=11 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:41:24.920187 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02ed218-0523-427f-8024-ffae772ca87b None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:41:25.001626 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02ed218-0523-427f-8024-ffae772ca87b None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:41:25.007995 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288919c-fca8-4fa4-aed8-cd012221e8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:41:25.009204 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288919c-fca8-4fa4-aed8-cd012221e8ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:41:25.009525 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288919c-fca8-4fa4-aed8-cd012221e8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:41:25.009806 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288919c-fca8-4fa4-aed8-cd012221e8ab admin admin] PATH_INFO: `/v3/users/14681b91d11746e6951d47e5d31e4d0a` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:41:25.011852 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b288919c-fca8-4fa4-aed8-cd012221e8ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=14681b91d11746e6951d47e5d31e4d0a)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:41:25.014831 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b288919c-fca8-4fa4-aed8-cd012221e8ab admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:41:25.087487 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b288919c-fca8-4fa4-aed8-cd012221e8ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14681b91d11746e6951d47e5d31e4d0a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:41:25.088565 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b288919c-fca8-4fa4-aed8-cd012221e8ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14681b91d11746e6951d47e5d31e4d0a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:41:25.109280 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b288919c-fca8-4fa4-aed8-cd012221e8ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14681b91d11746e6951d47e5d31e4d0a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:41:25.119589 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b288919c-fca8-4fa4-aed8-cd012221e8ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14681b91d11746e6951d47e5d31e4d0a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:41:25.132938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b288919c-fca8-4fa4-aed8-cd012221e8ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14681b91d11746e6951d47e5d31e4d0a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:41:25.141426 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1249/2538] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:41:24 2022] DELETE /identity/v3/users/14681b91d11746e6951d47e5d31e4d0a => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:41:25.147605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67800404-cbef-4615-9220-82375aa779f8 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:41:25.148523 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67800404-cbef-4615-9220-82375aa779f8 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:41:25.148747 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67800404-cbef-4615-9220-82375aa779f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:41:25.170471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-67800404-cbef-4615-9220-82375aa779f8 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:41:25.174311 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-67800404-cbef-4615-9220-82375aa779f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:41:25.253490 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1290/2539] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:41:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:41:25.264078 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288919c-fca8-4fa4-aed8-cd012221e8ab admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:41:25.266315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b288919c-fca8-4fa4-aed8-cd012221e8ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:41:25.341161 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288919c-fca8-4fa4-aed8-cd012221e8ab admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:41:25.346681 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d9e7d-7191-434e-b34b-9503bbfcba89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:41:25.347554 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d9e7d-7191-434e-b34b-9503bbfcba89 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:41:25.347737 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d9e7d-7191-434e-b34b-9503bbfcba89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:41:25.348819 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d9e7d-7191-434e-b34b-9503bbfcba89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:41:25.368123 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862d9e7d-7191-434e-b34b-9503bbfcba89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:41:25.369700 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862d9e7d-7191-434e-b34b-9503bbfcba89 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:41:25.382776 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1250/2540] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:41:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:41:25.813142 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67800404-cbef-4615-9220-82375aa779f8 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:41:25.815789 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-67800404-cbef-4615-9220-82375aa779f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:41:25.835032 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67800404-cbef-4615-9220-82375aa779f8 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:41:25.842006 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb30612-fabf-4a2f-ab77-143d32924670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:41:25.843207 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb30612-fabf-4a2f-ab77-143d32924670 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:41:25.843641 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb30612-fabf-4a2f-ab77-143d32924670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:41:25.844108 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb30612-fabf-4a2f-ab77-143d32924670 admin admin] PATH_INFO: `/v3/projects/62e59f8182c84c899ca2645d002625da` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:41:25.846113 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb30612-fabf-4a2f-ab77-143d32924670 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62e59f8182c84c899ca2645d002625da)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:41:25.848903 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb30612-fabf-4a2f-ab77-143d32924670 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:41:25.929848 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0fb30612-fabf-4a2f-ab77-143d32924670 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62e59f8182c84c899ca2645d002625da', 'request_id': 'req-0fb30612-fabf-4a2f-ab77-143d32924670'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:41:25.937684 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0fb30612-fabf-4a2f-ab77-143d32924670 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62e59f8182c84c899ca2645d002625da', 'request_id': 'req-0fb30612-fabf-4a2f-ab77-143d32924670'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:41:25.944583 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0fb30612-fabf-4a2f-ab77-143d32924670 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62e59f8182c84c899ca2645d002625da', 'request_id': 'req-0fb30612-fabf-4a2f-ab77-143d32924670'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:41:25.968256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0fb30612-fabf-4a2f-ab77-143d32924670 admin admin] The token cache is being invalidate because project 62e59f8182c84c899ca2645d002625da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:41:25.968807 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0fb30612-fabf-4a2f-ab77-143d32924670 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:41:25.969936 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0fb30612-fabf-4a2f-ab77-143d32924670 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:41:25.972634 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1291/2541] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:41:25 2022] DELETE /identity/v3/projects/62e59f8182c84c899ca2645d002625da => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:41:25.978367 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8e15f-9696-407c-acec-d8918ff3a66b None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:41:25.979120 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8e15f-9696-407c-acec-d8918ff3a66b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:41:25.979120 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8e15f-9696-407c-acec-d8918ff3a66b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:41:26.000600 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-6ca8e15f-9696-407c-acec-d8918ff3a66b None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:41:26.001988 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6ca8e15f-9696-407c-acec-d8918ff3a66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:41:26.034689 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:41:26.114143 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1251/2542] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:41:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:41:26.123864 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51e40b-bf7d-4755-9d12-16f01ec3d702 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:41:26.124124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51e40b-bf7d-4755-9d12-16f01ec3d702 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:41:26.124539 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51e40b-bf7d-4755-9d12-16f01ec3d702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:41:26.146356 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0a51e40b-bf7d-4755-9d12-16f01ec3d702 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:41:26.147982 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0a51e40b-bf7d-4755-9d12-16f01ec3d702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:41:26.159430 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1292/2543] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:41:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:41:26.166115 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8e15f-9696-407c-acec-d8918ff3a66b None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:41:26.185548 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8e15f-9696-407c-acec-d8918ff3a66b None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:41:26.194850 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892173a2-0275-456d-a42c-ff075efa3b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:41:26.196049 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892173a2-0275-456d-a42c-ff075efa3b32 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:41:26.196530 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892173a2-0275-456d-a42c-ff075efa3b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:41:26.197000 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892173a2-0275-456d-a42c-ff075efa3b32 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:41:26.198506 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892173a2-0275-456d-a42c-ff075efa3b32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:41:26.199763 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892173a2-0275-456d-a42c-ff075efa3b32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:41:26.202716 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892173a2-0275-456d-a42c-ff075efa3b32 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:41:26.229043 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1252/2544] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:41:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:41:26.234939 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51e40b-bf7d-4755-9d12-16f01ec3d702 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:41:26.247410 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51e40b-bf7d-4755-9d12-16f01ec3d702 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:41:26.253651 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db896ff0-d0fe-4b97-acb0-0e17e29afa40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:41:26.254420 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db896ff0-d0fe-4b97-acb0-0e17e29afa40 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:41:26.254711 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db896ff0-d0fe-4b97-acb0-0e17e29afa40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:41:26.255604 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db896ff0-d0fe-4b97-acb0-0e17e29afa40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:41:26.256388 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db896ff0-d0fe-4b97-acb0-0e17e29afa40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:41:26.258687 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db896ff0-d0fe-4b97-acb0-0e17e29afa40 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:41:26.310518 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1293/2545] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:41:26 2022] POST /identity/v3/projects => generated 363 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:41:26.313048 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892173a2-0275-456d-a42c-ff075efa3b32 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:41:26.326905 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892173a2-0275-456d-a42c-ff075efa3b32 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:41:26.420412 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd6ab9-0989-4cb0-bd6a-a9577270e1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:41:26.421409 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd6ab9-0989-4cb0-bd6a-a9577270e1fc admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:41:26.421939 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd6ab9-0989-4cb0-bd6a-a9577270e1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:41:26.422466 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd6ab9-0989-4cb0-bd6a-a9577270e1fc admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:41:26.423639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fd6ab9-0989-4cb0-bd6a-a9577270e1fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:41:26.425741 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fd6ab9-0989-4cb0-bd6a-a9577270e1fc admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:41:26.454426 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1253/2546] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:41:26 2022] POST /identity/v3/users => generated 282 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:41:26.466747 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db896ff0-d0fe-4b97-acb0-0e17e29afa40 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:41:26.484760 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db896ff0-d0fe-4b97-acb0-0e17e29afa40 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:41:26.492688 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c43672-b55b-4467-847b-ad297e69b59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:41:26.493870 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c43672-b55b-4467-847b-ad297e69b59f admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:41:26.494396 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c43672-b55b-4467-847b-ad297e69b59f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:41:26.494835 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c43672-b55b-4467-847b-ad297e69b59f admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:41:26.496735 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c43672-b55b-4467-847b-ad297e69b59f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:41:26.499386 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c43672-b55b-4467-847b-ad297e69b59f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:41:26.523536 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1294/2547] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:41:26.526815 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd6ab9-0989-4cb0-bd6a-a9577270e1fc admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:41:26.545387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd6ab9-0989-4cb0-bd6a-a9577270e1fc admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:41:26.554477 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e5d47-eda2-4e20-9a1a-06e6a89e0d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:41:26.555645 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e5d47-eda2-4e20-9a1a-06e6a89e0d26 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:41:26.556129 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e5d47-eda2-4e20-9a1a-06e6a89e0d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:41:26.556629 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e5d47-eda2-4e20-9a1a-06e6a89e0d26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:41:26.558113 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e5d47-eda2-4e20-9a1a-06e6a89e0d26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:41:26.561643 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e5d47-eda2-4e20-9a1a-06e6a89e0d26 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:41:26.594879 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1254/2548] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:41:26.599827 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c43672-b55b-4467-847b-ad297e69b59f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:41:26.612171 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c43672-b55b-4467-847b-ad297e69b59f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:41:26.619827 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8197e3db-2da1-4825-bdc5-82c6da72e5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:41:26.620975 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197e3db-2da1-4825-bdc5-82c6da72e5af admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:41:26.621155 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197e3db-2da1-4825-bdc5-82c6da72e5af admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:41:26.621557 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197e3db-2da1-4825-bdc5-82c6da72e5af admin admin] PATH_INFO: `/v3/projects/07f5d69d6dce41fe81f02fbce30594d7/users/9a1ce11b344b415abaf39841dab4772a/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:41:26.645324 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8197e3db-2da1-4825-bdc5-82c6da72e5af admin admin] RBAC: Authorizing `identity:create_grant(project_id=07f5d69d6dce41fe81f02fbce30594d7, user_id=9a1ce11b344b415abaf39841dab4772a, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:41:26.650221 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8197e3db-2da1-4825-bdc5-82c6da72e5af admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:41:26.674325 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1295/2549] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:41:26 2022] PUT /identity/v3/projects/07f5d69d6dce41fe81f02fbce30594d7/users/9a1ce11b344b415abaf39841dab4772a/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:41:26.687485 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3c192-4c07-4eab-8f5a-1121e138a85d None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:41:26.687852 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3c192-4c07-4eab-8f5a-1121e138a85d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:41:26.688296 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3c192-4c07-4eab-8f5a-1121e138a85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:41:26.723129 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-51e3c192-4c07-4eab-8f5a-1121e138a85d None None] MFA Rules not processed for user `9a1ce11b344b415abaf39841dab4772a`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:41:26.724546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-51e3c192-4c07-4eab-8f5a-1121e138a85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:41:26.843099 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1255/2550] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:41:26 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:41:26.857947 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8197e3db-2da1-4825-bdc5-82c6da72e5af admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:41:26.860221 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8197e3db-2da1-4825-bdc5-82c6da72e5af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:41:26.886384 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:41:26.939128 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8197e3db-2da1-4825-bdc5-82c6da72e5af admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:41:26.943382 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea4add1-6fe2-4cc8-a018-68c31d9b3661 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:41:26.944230 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea4add1-6fe2-4cc8-a018-68c31d9b3661 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:41:26.944698 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea4add1-6fe2-4cc8-a018-68c31d9b3661 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:41:26.945041 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea4add1-6fe2-4cc8-a018-68c31d9b3661 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:41:26.956883 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea4add1-6fe2-4cc8-a018-68c31d9b3661 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:41:26.958646 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea4add1-6fe2-4cc8-a018-68c31d9b3661 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:41:26.970858 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1296/2551] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:41:26 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:41:36.837719 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:41:36.965036 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=17 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:41:45.901266 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68a537-914c-4a47-96c0-f18d451c8e1f None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:41:45.902119 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68a537-914c-4a47-96c0-f18d451c8e1f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:41:45.903556 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68a537-914c-4a47-96c0-f18d451c8e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:41:45.924927 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-2b68a537-914c-4a47-96c0-f18d451c8e1f None None] MFA Rules not processed for user `eba91dff7cac401e81f6fb6391874af1`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:41:45.926045 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-2b68a537-914c-4a47-96c0-f18d451c8e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:41:45.998445 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1256/2552] 10.222.0.65 () {58 vars in 1009 bytes} [Thu May 19 21:41:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:41:46.054806 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea4add1-6fe2-4cc8-a018-68c31d9b3661 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:41:46.056305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0ea4add1-6fe2-4cc8-a018-68c31d9b3661 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:41:46.150176 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea4add1-6fe2-4cc8-a018-68c31d9b3661 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:41:46.156884 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bfe38d-c278-4828-8887-61ac69382fdd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e07984b36934aff9052541ef2a56f36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:41:46.157668 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bfe38d-c278-4828-8887-61ac69382fdd service swift] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:41:46.157980 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bfe38d-c278-4828-8887-61ac69382fdd service swift] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:41:46.158383 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bfe38d-c278-4828-8887-61ac69382fdd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:41:46.169662 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bfe38d-c278-4828-8887-61ac69382fdd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:41:46.171316 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bfe38d-c278-4828-8887-61ac69382fdd service swift] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:41:46.180134 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1297/2553] 10.222.0.65 () {60 vars in 1384 bytes} [Thu May 19 21:41:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 21:41:55.988912 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=25 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:41:56.176564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=32 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:42:03.915163 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d91f46-edc7-49b1-a243-b066e5c4704b None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:42:03.915964 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d91f46-edc7-49b1-a243-b066e5c4704b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:42:03.915964 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d91f46-edc7-49b1-a243-b066e5c4704b None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:42:03.917363 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1257/2554] 10.222.0.65 () {54 vars in 921 bytes} [Thu May 19 21:42:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:42:03.924474 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b121f002-d6f6-4ad0-9848-f6d7025ead68 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:42:03.925089 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b121f002-d6f6-4ad0-9848-f6d7025ead68 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:42:03.925089 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b121f002-d6f6-4ad0-9848-f6d7025ead68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:42:03.944672 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b121f002-d6f6-4ad0-9848-f6d7025ead68 None None] MFA Rules not processed for user `e254671b21fe4661a381b9521810cf80`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:42:03.945950 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b121f002-d6f6-4ad0-9848-f6d7025ead68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:42:04.030896 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1298/2555] 10.222.0.65 () {58 vars in 1018 bytes} [Thu May 19 21:42:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:42:04.044161 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d91f46-edc7-49b1-a243-b066e5c4704b None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:42:04.046562 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-94d91f46-edc7-49b1-a243-b066e5c4704b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:42:04.117991 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d91f46-edc7-49b1-a243-b066e5c4704b None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:42:04.122830 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a0f9e-75f6-4868-9d08-e10e4c0a53b8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea874ca4088d4e6c83c9d0497323cf6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:42:04.123558 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a0f9e-75f6-4868-9d08-e10e4c0a53b8 service glance] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:42:04.123849 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a0f9e-75f6-4868-9d08-e10e4c0a53b8 service glance] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:42:04.124122 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a0f9e-75f6-4868-9d08-e10e4c0a53b8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:42:04.135905 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a0f9e-75f6-4868-9d08-e10e4c0a53b8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:42:04.137479 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a0f9e-75f6-4868-9d08-e10e4c0a53b8 service glance] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:42:04.150001 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1258/2556] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:42:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:42:04.236830 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0851cb-0843-43c6-82a4-224c793e2407 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:42:04.237105 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0851cb-0843-43c6-82a4-224c793e2407 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:42:04.237403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0851cb-0843-43c6-82a4-224c793e2407 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:42:04.239120 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1299/2557] 10.222.0.65 () {54 vars in 921 bytes} [Thu May 19 21:42:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:42:04.243907 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42338755-28d8-418e-a859-1b5d2772f503 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:42:04.244409 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42338755-28d8-418e-a859-1b5d2772f503 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:42:04.244705 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42338755-28d8-418e-a859-1b5d2772f503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:42:04.269269 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-42338755-28d8-418e-a859-1b5d2772f503 None None] MFA Rules not processed for user `2201e66caa4b4a57be09d845d7cb0023`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:42:04.270499 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-42338755-28d8-418e-a859-1b5d2772f503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:42:04.328242 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1259/2558] 10.222.0.65 () {58 vars in 1018 bytes} [Thu May 19 21:42:04 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:42:04.340363 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0851cb-0843-43c6-82a4-224c793e2407 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:42:04.342773 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9c0851cb-0843-43c6-82a4-224c793e2407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:42:04.358526 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0851cb-0843-43c6-82a4-224c793e2407 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:42:04.362619 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facfe55a-8422-4afa-b198-0207375aa4a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:42:04.363268 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facfe55a-8422-4afa-b198-0207375aa4a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:42:04.363551 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facfe55a-8422-4afa-b198-0207375aa4a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:42:04.363828 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facfe55a-8422-4afa-b198-0207375aa4a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:42:04.371693 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facfe55a-8422-4afa-b198-0207375aa4a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:42:04.373672 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facfe55a-8422-4afa-b198-0207375aa4a1 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:42:04.381627 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1300/2559] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:42:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:42:04.467754 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796aeb0-c7d8-4d57-83f4-9cf58ece7369 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:42:04.468039 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796aeb0-c7d8-4d57-83f4-9cf58ece7369 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:42:04.468341 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796aeb0-c7d8-4d57-83f4-9cf58ece7369 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:42:04.469744 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1260/2560] 10.222.0.65 () {54 vars in 921 bytes} [Thu May 19 21:42:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:42:04.475655 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d26b5b-0a11-4732-85fe-f05ec40e6ce1 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:42:04.475942 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d26b5b-0a11-4732-85fe-f05ec40e6ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:42:04.476217 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d26b5b-0a11-4732-85fe-f05ec40e6ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:42:04.497373 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-10d26b5b-0a11-4732-85fe-f05ec40e6ce1 None None] MFA Rules not processed for user `e254671b21fe4661a381b9521810cf80`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:42:04.498611 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-10d26b5b-0a11-4732-85fe-f05ec40e6ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:42:04.509087 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1301/2561] 10.222.0.65 () {58 vars in 1018 bytes} [Thu May 19 21:42:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:42:04.520642 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796aeb0-c7d8-4d57-83f4-9cf58ece7369 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:42:04.522716 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4796aeb0-c7d8-4d57-83f4-9cf58ece7369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:42:04.538489 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796aeb0-c7d8-4d57-83f4-9cf58ece7369 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:42:04.542631 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00024698-83ee-4986-b260-0ff177adfa87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:42:04.543270 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00024698-83ee-4986-b260-0ff177adfa87 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:42:04.543541 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00024698-83ee-4986-b260-0ff177adfa87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:42:04.543819 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00024698-83ee-4986-b260-0ff177adfa87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:42:04.554893 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00024698-83ee-4986-b260-0ff177adfa87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:42:04.556309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00024698-83ee-4986-b260-0ff177adfa87 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:42:04.567124 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1261/2562] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:42:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:42:06.260126 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9183f-5dd0-4192-ad84-1852b73fcb6b None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:42:06.260555 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9183f-5dd0-4192-ad84-1852b73fcb6b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:42:06.260653 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9183f-5dd0-4192-ad84-1852b73fcb6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:42:06.279568 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-56e9183f-5dd0-4192-ad84-1852b73fcb6b None None] MFA Rules not processed for user `eba91dff7cac401e81f6fb6391874af1`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:42:06.280774 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-56e9183f-5dd0-4192-ad84-1852b73fcb6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:42:06.288269 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1302/2563] 10.222.0.65 () {58 vars in 1009 bytes} [Thu May 19 21:42:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:42:06.300855 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00024698-83ee-4986-b260-0ff177adfa87 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:42:06.313088 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00024698-83ee-4986-b260-0ff177adfa87 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:42:06.319894 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a279ab01-0633-4656-8348-c223a793d648 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e07984b36934aff9052541ef2a56f36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:42:06.320523 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279ab01-0633-4656-8348-c223a793d648 service swift] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:42:06.320864 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279ab01-0633-4656-8348-c223a793d648 service swift] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:42:06.321145 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279ab01-0633-4656-8348-c223a793d648 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:42:06.330831 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a279ab01-0633-4656-8348-c223a793d648 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:42:06.332437 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a279ab01-0633-4656-8348-c223a793d648 service swift] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:42:06.340914 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1262/2564] 10.222.0.65 () {60 vars in 1384 bytes} [Thu May 19 21:42:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 21:42:12.291046 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e9183f-5dd0-4192-ad84-1852b73fcb6b None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:42:12.302177 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e9183f-5dd0-4192-ad84-1852b73fcb6b None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:42:12.306846 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f22aea-a614-40aa-8479-6c8346421019 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:42:12.307500 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f22aea-a614-40aa-8479-6c8346421019 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:42:12.307782 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f22aea-a614-40aa-8479-6c8346421019 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:42:12.308717 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f22aea-a614-40aa-8479-6c8346421019 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:42:12.311109 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d6f22aea-a614-40aa-8479-6c8346421019 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:42:12.326752 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f22aea-a614-40aa-8479-6c8346421019 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:42:12.328007 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f22aea-a614-40aa-8479-6c8346421019 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:42:12.340124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1303/2565] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:42:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:42:13.553012 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a279ab01-0633-4656-8348-c223a793d648 service swift] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:42:13.562812 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a279ab01-0633-4656-8348-c223a793d648 service swift] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:42:13.566646 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a5715-b286-4b6b-98db-6cbc7123f60f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:42:13.567322 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a5715-b286-4b6b-98db-6cbc7123f60f service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:42:13.567604 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a5715-b286-4b6b-98db-6cbc7123f60f service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:42:13.567875 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a5715-b286-4b6b-98db-6cbc7123f60f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:42:13.569740 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-928a5715-b286-4b6b-98db-6cbc7123f60f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:42:13.584774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928a5715-b286-4b6b-98db-6cbc7123f60f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:42:13.586211 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928a5715-b286-4b6b-98db-6cbc7123f60f service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:42:13.598375 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1263/2566] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:42:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:42:15.749825 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f22aea-a614-40aa-8479-6c8346421019 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:42:15.837378 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f22aea-a614-40aa-8479-6c8346421019 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:42:15.843177 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cff9a9-3001-407b-a6ce-b72cb6e45bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:42:15.844584 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cff9a9-3001-407b-a6ce-b72cb6e45bbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:42:15.844872 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cff9a9-3001-407b-a6ce-b72cb6e45bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:42:15.845159 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cff9a9-3001-407b-a6ce-b72cb6e45bbb admin admin] PATH_INFO: `/v3/users/9a1ce11b344b415abaf39841dab4772a` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:42:15.846532 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cff9a9-3001-407b-a6ce-b72cb6e45bbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a1ce11b344b415abaf39841dab4772a)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:42:15.848495 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cff9a9-3001-407b-a6ce-b72cb6e45bbb admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:42:15.902495 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-62cff9a9-3001-407b-a6ce-b72cb6e45bbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a1ce11b344b415abaf39841dab4772a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:42:15.909929 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:42:15.918294 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-62cff9a9-3001-407b-a6ce-b72cb6e45bbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a1ce11b344b415abaf39841dab4772a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:42:15.919005 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-62cff9a9-3001-407b-a6ce-b72cb6e45bbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a1ce11b344b415abaf39841dab4772a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:42:15.919620 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-62cff9a9-3001-407b-a6ce-b72cb6e45bbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a1ce11b344b415abaf39841dab4772a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:42:15.926730 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-62cff9a9-3001-407b-a6ce-b72cb6e45bbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a1ce11b344b415abaf39841dab4772a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:42:15.942182 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1304/2567] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:42:15 2022] DELETE /identity/v3/users/9a1ce11b344b415abaf39841dab4772a => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:42:15.946993 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a85510f-df2d-4a1a-8bfa-40f33b2a66d0 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:42:15.947503 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a85510f-df2d-4a1a-8bfa-40f33b2a66d0 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:42:15.947613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a85510f-df2d-4a1a-8bfa-40f33b2a66d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:42:15.968037 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-9a85510f-df2d-4a1a-8bfa-40f33b2a66d0 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:42:15.969332 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9a85510f-df2d-4a1a-8bfa-40f33b2a66d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:42:16.084016 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1264/2568] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:42:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:42:16.091053 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cff9a9-3001-407b-a6ce-b72cb6e45bbb admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:42:16.092605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-62cff9a9-3001-407b-a6ce-b72cb6e45bbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:42:16.166462 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cff9a9-3001-407b-a6ce-b72cb6e45bbb admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:42:16.171126 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94477d8d-909f-4bb5-a193-a3cb29abaa6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:42:16.171887 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94477d8d-909f-4bb5-a193-a3cb29abaa6c service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:42:16.172318 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94477d8d-909f-4bb5-a193-a3cb29abaa6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:42:16.172622 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94477d8d-909f-4bb5-a193-a3cb29abaa6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:42:16.184543 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94477d8d-909f-4bb5-a193-a3cb29abaa6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:42:16.186020 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94477d8d-909f-4bb5-a193-a3cb29abaa6c service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:42:16.197385 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1305/2569] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:42:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:42:16.496458 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a85510f-df2d-4a1a-8bfa-40f33b2a66d0 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:42:16.498087 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9a85510f-df2d-4a1a-8bfa-40f33b2a66d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:42:16.516668 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a85510f-df2d-4a1a-8bfa-40f33b2a66d0 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:42:16.530140 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a0be6f-5e6b-4094-82b4-30cb29ecf376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:42:16.532178 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0be6f-5e6b-4094-82b4-30cb29ecf376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:42:16.532753 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0be6f-5e6b-4094-82b4-30cb29ecf376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:42:16.533192 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0be6f-5e6b-4094-82b4-30cb29ecf376 admin admin] PATH_INFO: `/v3/projects/07f5d69d6dce41fe81f02fbce30594d7` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:42:16.534987 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a0be6f-5e6b-4094-82b4-30cb29ecf376 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07f5d69d6dce41fe81f02fbce30594d7)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:42:16.537519 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a0be6f-5e6b-4094-82b4-30cb29ecf376 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:42:16.549493 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:42:16.598813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-61a0be6f-5e6b-4094-82b4-30cb29ecf376 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07f5d69d6dce41fe81f02fbce30594d7', 'request_id': 'req-61a0be6f-5e6b-4094-82b4-30cb29ecf376'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:42:16.605430 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-61a0be6f-5e6b-4094-82b4-30cb29ecf376 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07f5d69d6dce41fe81f02fbce30594d7', 'request_id': 'req-61a0be6f-5e6b-4094-82b4-30cb29ecf376'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:42:16.614175 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-61a0be6f-5e6b-4094-82b4-30cb29ecf376 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07f5d69d6dce41fe81f02fbce30594d7', 'request_id': 'req-61a0be6f-5e6b-4094-82b4-30cb29ecf376'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:42:16.625632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-61a0be6f-5e6b-4094-82b4-30cb29ecf376 admin admin] The token cache is being invalidate because project 07f5d69d6dce41fe81f02fbce30594d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:42:16.625935 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-61a0be6f-5e6b-4094-82b4-30cb29ecf376 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:42:16.626715 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-61a0be6f-5e6b-4094-82b4-30cb29ecf376 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:42:16.628299 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1265/2570] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:42:16 2022] DELETE /identity/v3/projects/07f5d69d6dce41fe81f02fbce30594d7 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:42:20.056464 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1556232-c369-4fb6-9bef-c2d2b9da5ba8 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:42:20.057039 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1556232-c369-4fb6-9bef-c2d2b9da5ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:42:20.057039 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1556232-c369-4fb6-9bef-c2d2b9da5ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:42:20.076772 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b1556232-c369-4fb6-9bef-c2d2b9da5ba8 None None] MFA Rules not processed for user `0a58f82117ab437cab5607c204c86547`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:42:20.078021 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b1556232-c369-4fb6-9bef-c2d2b9da5ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:42:20.137979 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1306/2571] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:42:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:42:20.629841 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a0be6f-5e6b-4094-82b4-30cb29ecf376 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:42:20.631415 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-61a0be6f-5e6b-4094-82b4-30cb29ecf376 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:42:20.752600 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a0be6f-5e6b-4094-82b4-30cb29ecf376 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:42:20.756375 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbeb801-b1da-4c86-bdfc-d9563bfa1138 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:42:20.757149 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbeb801-b1da-4c86-bdfc-d9563bfa1138 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:42:20.757433 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbeb801-b1da-4c86-bdfc-d9563bfa1138 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:42:20.757710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbeb801-b1da-4c86-bdfc-d9563bfa1138 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:42:20.759701 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8cbeb801-b1da-4c86-bdfc-d9563bfa1138 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:42:20.771813 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbeb801-b1da-4c86-bdfc-d9563bfa1138 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:42:20.773441 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbeb801-b1da-4c86-bdfc-d9563bfa1138 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:42:20.783253 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1266/2572] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:42:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:42:27.931546 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1556232-c369-4fb6-9bef-c2d2b9da5ba8 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:42:27.933152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b1556232-c369-4fb6-9bef-c2d2b9da5ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:42:28.023388 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1556232-c369-4fb6-9bef-c2d2b9da5ba8 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:42:28.029547 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1aff5e-af2f-42f5-ba05-17791124a766 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:42:28.030354 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1aff5e-af2f-42f5-ba05-17791124a766 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:42:28.034764 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1aff5e-af2f-42f5-ba05-17791124a766 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:42:28.035204 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1aff5e-af2f-42f5-ba05-17791124a766 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:42:28.038106 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ba1aff5e-af2f-42f5-ba05-17791124a766 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:42:28.056314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:42:28.163509 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1aff5e-af2f-42f5-ba05-17791124a766 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:42:28.165384 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1aff5e-af2f-42f5-ba05-17791124a766 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:42:28.190322 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1307/2573] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:42:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:42:28.928604 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbeb801-b1da-4c86-bdfc-d9563bfa1138 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:42:28.938419 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbeb801-b1da-4c86-bdfc-d9563bfa1138 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:42:28.942897 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01952cc8-0e37-4cfb-b964-5b277afbaab1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:42:28.943609 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01952cc8-0e37-4cfb-b964-5b277afbaab1 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:42:28.943883 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01952cc8-0e37-4cfb-b964-5b277afbaab1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:42:28.944352 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01952cc8-0e37-4cfb-b964-5b277afbaab1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:42:28.947146 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-01952cc8-0e37-4cfb-b964-5b277afbaab1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:42:28.959640 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01952cc8-0e37-4cfb-b964-5b277afbaab1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:42:28.961104 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01952cc8-0e37-4cfb-b964-5b277afbaab1 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:42:28.971944 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1267/2574] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:42:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:42:33.822734 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823641d-cbac-4621-a8bf-8016699b3146 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:42:33.823595 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823641d-cbac-4621-a8bf-8016699b3146 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:42:33.823700 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823641d-cbac-4621-a8bf-8016699b3146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:42:33.842011 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0823641d-cbac-4621-a8bf-8016699b3146 None None] MFA Rules not processed for user `eba91dff7cac401e81f6fb6391874af1`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:42:33.843913 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0823641d-cbac-4621-a8bf-8016699b3146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:42:33.907253 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1308/2575] 10.222.0.65 () {58 vars in 967 bytes} [Thu May 19 21:42:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:42:33.921181 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01952cc8-0e37-4cfb-b964-5b277afbaab1 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:42:33.922766 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-01952cc8-0e37-4cfb-b964-5b277afbaab1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:42:34.007831 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01952cc8-0e37-4cfb-b964-5b277afbaab1 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:42:34.014126 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9339e1b0-f1a9-45bf-9ac9-c17c57488e4f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7e07984b36934aff9052541ef2a56f36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:42:34.014867 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339e1b0-f1a9-45bf-9ac9-c17c57488e4f service swift] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:42:34.015150 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339e1b0-f1a9-45bf-9ac9-c17c57488e4f service swift] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:42:34.015424 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339e1b0-f1a9-45bf-9ac9-c17c57488e4f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:42:34.025343 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9339e1b0-f1a9-45bf-9ac9-c17c57488e4f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:42:34.027072 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9339e1b0-f1a9-45bf-9ac9-c17c57488e4f service swift] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:42:34.036648 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1268/2576] 10.222.0.65 () {60 vars in 1384 bytes} [Thu May 19 21:42:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 21:42:43.900687 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=58 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:42:44.032545 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=85,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:42:47.178068 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0823641d-cbac-4621-a8bf-8016699b3146 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:42:47.179632 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0823641d-cbac-4621-a8bf-8016699b3146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:42:47.194143 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0823641d-cbac-4621-a8bf-8016699b3146 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:42:47.200403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd074ef5-c4a7-4bfa-b09b-3d9419b728cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:42:47.201099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd074ef5-c4a7-4bfa-b09b-3d9419b728cf service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:42:47.201382 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd074ef5-c4a7-4bfa-b09b-3d9419b728cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:42:47.201656 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd074ef5-c4a7-4bfa-b09b-3d9419b728cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:42:47.203632 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-dd074ef5-c4a7-4bfa-b09b-3d9419b728cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:42:47.279928 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd074ef5-c4a7-4bfa-b09b-3d9419b728cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:42:47.281391 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd074ef5-c4a7-4bfa-b09b-3d9419b728cf service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:42:47.292859 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1309/2577] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:42:47 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:42:57.287167 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=30 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:43:26.258255 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9339e1b0-f1a9-45bf-9ac9-c17c57488e4f service swift] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:43:26.269195 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9339e1b0-f1a9-45bf-9ac9-c17c57488e4f service swift] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:43:26.273823 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e55f7f-69ce-4c65-9143-74ad3f8d5697 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:43:26.274530 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e55f7f-69ce-4c65-9143-74ad3f8d5697 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:43:26.274810 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e55f7f-69ce-4c65-9143-74ad3f8d5697 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:43:26.275082 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e55f7f-69ce-4c65-9143-74ad3f8d5697 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:43:26.278311 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e0e55f7f-69ce-4c65-9143-74ad3f8d5697 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:43:26.353352 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e55f7f-69ce-4c65-9143-74ad3f8d5697 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:43:26.354867 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e55f7f-69ce-4c65-9143-74ad3f8d5697 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:43:26.369068 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1269/2578] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:43:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:43:29.792021 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd074ef5-c4a7-4bfa-b09b-3d9419b728cf service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:43:29.793732 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-dd074ef5-c4a7-4bfa-b09b-3d9419b728cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:43:29.809702 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd074ef5-c4a7-4bfa-b09b-3d9419b728cf service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:43:29.814690 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9678f612-8ff7-421d-b22e-70ed1fff5652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:43:29.815669 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678f612-8ff7-421d-b22e-70ed1fff5652 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:43:29.815940 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678f612-8ff7-421d-b22e-70ed1fff5652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:43:29.816221 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678f612-8ff7-421d-b22e-70ed1fff5652 admin admin] PATH_INFO: `/v3/users/6d16feb169ac46f7983a6b8045de77c9` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:43:29.831385 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9678f612-8ff7-421d-b22e-70ed1fff5652 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d16feb169ac46f7983a6b8045de77c9)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:43:29.833385 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9678f612-8ff7-421d-b22e-70ed1fff5652 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:43:29.887460 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9678f612-8ff7-421d-b22e-70ed1fff5652 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d16feb169ac46f7983a6b8045de77c9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:43:29.901255 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9678f612-8ff7-421d-b22e-70ed1fff5652 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d16feb169ac46f7983a6b8045de77c9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:43:29.901975 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9678f612-8ff7-421d-b22e-70ed1fff5652 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d16feb169ac46f7983a6b8045de77c9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:43:29.902616 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9678f612-8ff7-421d-b22e-70ed1fff5652 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d16feb169ac46f7983a6b8045de77c9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:43:29.910456 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9678f612-8ff7-421d-b22e-70ed1fff5652 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d16feb169ac46f7983a6b8045de77c9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:43:29.927559 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1310/2579] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:43:29 2022] DELETE /identity/v3/users/6d16feb169ac46f7983a6b8045de77c9 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:43:30.109132 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e55f7f-69ce-4c65-9143-74ad3f8d5697 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:43:30.121429 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e55f7f-69ce-4c65-9143-74ad3f8d5697 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:43:30.184657 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d17557-42db-4c51-8074-f445986355d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:43:30.185536 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d17557-42db-4c51-8074-f445986355d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:43:30.185818 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d17557-42db-4c51-8074-f445986355d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:43:30.186095 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d17557-42db-4c51-8074-f445986355d6 admin admin] PATH_INFO: `/v3/projects/4f5916770e9b443489fe7aa76d663fe1` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:43:30.197358 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d17557-42db-4c51-8074-f445986355d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f5916770e9b443489fe7aa76d663fe1)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:43:30.199344 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d17557-42db-4c51-8074-f445986355d6 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:43:30.253732 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-74d17557-42db-4c51-8074-f445986355d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f5916770e9b443489fe7aa76d663fe1', 'request_id': 'req-74d17557-42db-4c51-8074-f445986355d6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:43:30.260889 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-74d17557-42db-4c51-8074-f445986355d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f5916770e9b443489fe7aa76d663fe1', 'request_id': 'req-74d17557-42db-4c51-8074-f445986355d6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:43:30.269900 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-74d17557-42db-4c51-8074-f445986355d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f5916770e9b443489fe7aa76d663fe1', 'request_id': 'req-74d17557-42db-4c51-8074-f445986355d6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:43:30.281094 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-74d17557-42db-4c51-8074-f445986355d6 admin admin] The token cache is being invalidate because project 4f5916770e9b443489fe7aa76d663fe1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:43:30.281377 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-74d17557-42db-4c51-8074-f445986355d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:43:30.281996 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-74d17557-42db-4c51-8074-f445986355d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:43:30.283632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1270/2580] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:43:30 2022] DELETE /identity/v3/projects/4f5916770e9b443489fe7aa76d663fe1 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:43:30.291160 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523cde08-9368-40d2-a053-730939ca9725 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:43:30.291441 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523cde08-9368-40d2-a053-730939ca9725 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:43:30.291809 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523cde08-9368-40d2-a053-730939ca9725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:43:30.317732 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-523cde08-9368-40d2-a053-730939ca9725 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:43:30.318886 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-523cde08-9368-40d2-a053-730939ca9725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:43:30.384452 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1311/2581] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:43:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:43:30.395437 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec803dea-5847-4243-b617-1becd5282f39 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:43:30.395735 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec803dea-5847-4243-b617-1becd5282f39 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:43:30.396037 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec803dea-5847-4243-b617-1becd5282f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:43:30.414994 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-ec803dea-5847-4243-b617-1becd5282f39 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:43:30.416209 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ec803dea-5847-4243-b617-1becd5282f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:43:30.426551 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1271/2582] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:43:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:43:30.432330 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523cde08-9368-40d2-a053-730939ca9725 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:43:30.447933 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523cde08-9368-40d2-a053-730939ca9725 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:43:30.456263 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe1a5d6-8066-46d7-90e7-4b6757be8b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:43:30.457110 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe1a5d6-8066-46d7-90e7-4b6757be8b7d admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:43:30.457406 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe1a5d6-8066-46d7-90e7-4b6757be8b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:43:30.457676 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe1a5d6-8066-46d7-90e7-4b6757be8b7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:43:30.458452 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe1a5d6-8066-46d7-90e7-4b6757be8b7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:43:30.458765 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe1a5d6-8066-46d7-90e7-4b6757be8b7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:43:30.460375 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe1a5d6-8066-46d7-90e7-4b6757be8b7d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:43:30.473209 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1312/2583] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:43:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:43:30.477709 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec803dea-5847-4243-b617-1becd5282f39 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:43:30.488121 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec803dea-5847-4243-b617-1becd5282f39 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:43:30.493559 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65e055-747b-4573-ac4b-a4bdb49d45a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:43:30.494225 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65e055-747b-4573-ac4b-a4bdb49d45a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:43:30.494542 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65e055-747b-4573-ac4b-a4bdb49d45a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:43:30.494822 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65e055-747b-4573-ac4b-a4bdb49d45a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:43:30.495518 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd65e055-747b-4573-ac4b-a4bdb49d45a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:43:30.497451 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd65e055-747b-4573-ac4b-a4bdb49d45a3 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:43:30.502114 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:43:30.530745 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1272/2584] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:43:30 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:43:30.532047 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe1a5d6-8066-46d7-90e7-4b6757be8b7d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:43:30.543160 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe1a5d6-8066-46d7-90e7-4b6757be8b7d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:43:30.592474 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:43:30.607775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d152b-1a03-48f0-8857-7c2019be0eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:43:30.608459 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d152b-1a03-48f0-8857-7c2019be0eea admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:43:30.608744 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d152b-1a03-48f0-8857-7c2019be0eea admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:43:30.609021 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d152b-1a03-48f0-8857-7c2019be0eea admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:43:30.609721 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d152b-1a03-48f0-8857-7c2019be0eea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:43:30.611675 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d152b-1a03-48f0-8857-7c2019be0eea admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:43:30.636288 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1313/2585] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:43:30 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:43:30.641137 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65e055-747b-4573-ac4b-a4bdb49d45a3 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:43:30.652532 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65e055-747b-4573-ac4b-a4bdb49d45a3 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:43:30.659076 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07074cc4-1788-4061-b47e-e42c75900ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:43:30.660217 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07074cc4-1788-4061-b47e-e42c75900ae3 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:43:30.660706 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07074cc4-1788-4061-b47e-e42c75900ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:43:30.661140 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07074cc4-1788-4061-b47e-e42c75900ae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:43:30.662147 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07074cc4-1788-4061-b47e-e42c75900ae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:43:30.663799 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07074cc4-1788-4061-b47e-e42c75900ae3 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:43:30.674678 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1273/2586] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:43:30.680891 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d152b-1a03-48f0-8857-7c2019be0eea admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:43:30.696156 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d152b-1a03-48f0-8857-7c2019be0eea admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:43:30.701232 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b64314-a59a-4ba0-8f0a-83bc1a518375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:43:30.701902 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b64314-a59a-4ba0-8f0a-83bc1a518375 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:43:30.702181 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b64314-a59a-4ba0-8f0a-83bc1a518375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:43:30.702486 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b64314-a59a-4ba0-8f0a-83bc1a518375 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:43:30.703223 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b64314-a59a-4ba0-8f0a-83bc1a518375 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:43:30.704815 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b64314-a59a-4ba0-8f0a-83bc1a518375 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:43:30.715619 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1314/2587] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:43:30.720167 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07074cc4-1788-4061-b47e-e42c75900ae3 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:43:30.730543 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07074cc4-1788-4061-b47e-e42c75900ae3 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:43:30.735290 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1ac70-5a70-44cf-891c-891d32045807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:43:30.736182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1ac70-5a70-44cf-891c-891d32045807 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:43:30.736459 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1ac70-5a70-44cf-891c-891d32045807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:43:30.736727 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1ac70-5a70-44cf-891c-891d32045807 admin admin] PATH_INFO: `/v3/projects/9303188058ce403586f38325a1deaea0/users/25e86dd7ac714324a6ac8559be408dc7/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:43:30.752128 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d1ac70-5a70-44cf-891c-891d32045807 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9303188058ce403586f38325a1deaea0, user_id=25e86dd7ac714324a6ac8559be408dc7, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:43:30.754984 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d1ac70-5a70-44cf-891c-891d32045807 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:43:30.777031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1274/2588] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:43:30 2022] PUT /identity/v3/projects/9303188058ce403586f38325a1deaea0/users/25e86dd7ac714324a6ac8559be408dc7/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:43:30.790793 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6f261-1795-4a2a-99a7-ecbc34dc191f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:43:30.791088 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6f261-1795-4a2a-99a7-ecbc34dc191f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:43:30.791395 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6f261-1795-4a2a-99a7-ecbc34dc191f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:43:30.818664 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-10f6f261-1795-4a2a-99a7-ecbc34dc191f None None] MFA Rules not processed for user `25e86dd7ac714324a6ac8559be408dc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:43:30.819881 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-10f6f261-1795-4a2a-99a7-ecbc34dc191f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:43:30.889621 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1315/2589] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:43:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:43:30.896095 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1ac70-5a70-44cf-891c-891d32045807 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:43:30.961551 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1ac70-5a70-44cf-891c-891d32045807 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:43:30.967185 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851cb22e-ca61-463a-ac5b-04f6b344327a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:43:30.967889 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851cb22e-ca61-463a-ac5b-04f6b344327a admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:43:30.968171 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851cb22e-ca61-463a-ac5b-04f6b344327a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:43:30.969447 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851cb22e-ca61-463a-ac5b-04f6b344327a admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:43:30.970255 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851cb22e-ca61-463a-ac5b-04f6b344327a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:43:30.972386 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851cb22e-ca61-463a-ac5b-04f6b344327a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:43:31.003888 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1275/2590] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:43:30 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:43:31.008260 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f6f261-1795-4a2a-99a7-ecbc34dc191f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:43:31.020154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f6f261-1795-4a2a-99a7-ecbc34dc191f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:43:31.079757 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032d6d9-0bcb-46c9-98c5-2a6f10576a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:43:31.080460 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032d6d9-0bcb-46c9-98c5-2a6f10576a0e admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:43:31.080727 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032d6d9-0bcb-46c9-98c5-2a6f10576a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:43:31.081004 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032d6d9-0bcb-46c9-98c5-2a6f10576a0e admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:43:31.081831 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0032d6d9-0bcb-46c9-98c5-2a6f10576a0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:43:31.083900 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0032d6d9-0bcb-46c9-98c5-2a6f10576a0e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:43:31.108085 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1316/2591] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:43:31 2022] POST /identity/v3/users => generated 295 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:43:31.112932 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851cb22e-ca61-463a-ac5b-04f6b344327a admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:43:31.126761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851cb22e-ca61-463a-ac5b-04f6b344327a admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:43:31.132342 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a8816-8e92-412f-9091-9d47014f8609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:43:31.132794 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a8816-8e92-412f-9091-9d47014f8609 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:43:31.133162 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a8816-8e92-412f-9091-9d47014f8609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:43:31.133608 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a8816-8e92-412f-9091-9d47014f8609 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:43:31.134329 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84a8816-8e92-412f-9091-9d47014f8609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:43:31.135953 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84a8816-8e92-412f-9091-9d47014f8609 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:43:31.148971 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1276/2592] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:43:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:43:31.153951 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032d6d9-0bcb-46c9-98c5-2a6f10576a0e admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:43:31.165320 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032d6d9-0bcb-46c9-98c5-2a6f10576a0e admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:43:31.172376 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a630b-3fbc-43f1-9433-87085f053e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:43:31.173452 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a630b-3fbc-43f1-9433-87085f053e4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:43:31.173822 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a630b-3fbc-43f1-9433-87085f053e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:43:31.174189 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a630b-3fbc-43f1-9433-87085f053e4a admin admin] PATH_INFO: `/v3/projects/932f862d022d4ee8961662e071e806fb/users/668cc44b4bbc46b5bc24793f8d55e37d/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:43:31.190742 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182a630b-3fbc-43f1-9433-87085f053e4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=932f862d022d4ee8961662e071e806fb, user_id=668cc44b4bbc46b5bc24793f8d55e37d, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:43:31.193422 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182a630b-3fbc-43f1-9433-87085f053e4a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:43:31.210001 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1317/2593] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:43:31 2022] PUT /identity/v3/projects/932f862d022d4ee8961662e071e806fb/users/668cc44b4bbc46b5bc24793f8d55e37d/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:43:31.214480 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a8816-8e92-412f-9091-9d47014f8609 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:43:31.224924 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a8816-8e92-412f-9091-9d47014f8609 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:43:31.285415 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d32a973-7ebb-44e3-9d21-43424f3719f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:43:31.286092 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32a973-7ebb-44e3-9d21-43424f3719f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:43:31.286416 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32a973-7ebb-44e3-9d21-43424f3719f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:43:31.286693 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32a973-7ebb-44e3-9d21-43424f3719f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:43:31.287427 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d32a973-7ebb-44e3-9d21-43424f3719f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:43:31.289056 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d32a973-7ebb-44e3-9d21-43424f3719f5 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:43:31.300008 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1277/2594] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:43:31 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:43:31.304805 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a630b-3fbc-43f1-9433-87085f053e4a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:43:31.317032 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a630b-3fbc-43f1-9433-87085f053e4a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:43:31.322599 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba88eb5-5ace-475d-9da1-20d156b4f2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:43:31.323500 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba88eb5-5ace-475d-9da1-20d156b4f2c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:43:31.323778 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba88eb5-5ace-475d-9da1-20d156b4f2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:43:31.326514 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba88eb5-5ace-475d-9da1-20d156b4f2c2 admin admin] PATH_INFO: `/v3/projects/932f862d022d4ee8961662e071e806fb/users/668cc44b4bbc46b5bc24793f8d55e37d/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:43:31.333062 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba88eb5-5ace-475d-9da1-20d156b4f2c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=932f862d022d4ee8961662e071e806fb, user_id=668cc44b4bbc46b5bc24793f8d55e37d, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:43:31.335780 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba88eb5-5ace-475d-9da1-20d156b4f2c2 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:43:31.348709 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1318/2595] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:43:31 2022] PUT /identity/v3/projects/932f862d022d4ee8961662e071e806fb/users/668cc44b4bbc46b5bc24793f8d55e37d/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:43:31.360001 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f808dd-87ba-4ab8-8d2e-2f02b70104d4 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:43:31.360228 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f808dd-87ba-4ab8-8d2e-2f02b70104d4 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:43:31.360582 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f808dd-87ba-4ab8-8d2e-2f02b70104d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:43:31.389509 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-20f808dd-87ba-4ab8-8d2e-2f02b70104d4 None None] MFA Rules not processed for user `668cc44b4bbc46b5bc24793f8d55e37d`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:43:31.390782 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-20f808dd-87ba-4ab8-8d2e-2f02b70104d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:43:31.400559 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:43:31.466672 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1278/2596] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:43:31 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:43:31.478059 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba88eb5-5ace-475d-9da1-20d156b4f2c2 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:43:31.480009 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6ba88eb5-5ace-475d-9da1-20d156b4f2c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:43:31.492374 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:43:31.543698 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba88eb5-5ace-475d-9da1-20d156b4f2c2 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:43:31.548399 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e58a1-5dda-4f58-afb1-7f957fb0bd31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:43:31.549051 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e58a1-5dda-4f58-afb1-7f957fb0bd31 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:43:31.549333 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e58a1-5dda-4f58-afb1-7f957fb0bd31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:43:31.549612 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e58a1-5dda-4f58-afb1-7f957fb0bd31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:43:31.617655 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694e58a1-5dda-4f58-afb1-7f957fb0bd31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:43:31.619056 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694e58a1-5dda-4f58-afb1-7f957fb0bd31 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:43:31.629708 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1319/2597] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:43:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:43:38.350539 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f808dd-87ba-4ab8-8d2e-2f02b70104d4 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:43:38.360812 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f808dd-87ba-4ab8-8d2e-2f02b70104d4 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:43:38.364558 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58709687-56c4-4b1e-887a-44d25bf24b8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:43:38.365205 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58709687-56c4-4b1e-887a-44d25bf24b8f service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:43:38.365568 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58709687-56c4-4b1e-887a-44d25bf24b8f service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:43:38.365865 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58709687-56c4-4b1e-887a-44d25bf24b8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:43:38.375668 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58709687-56c4-4b1e-887a-44d25bf24b8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:43:38.377118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58709687-56c4-4b1e-887a-44d25bf24b8f service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:43:38.387202 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1279/2598] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:43:38 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:43:41.624527 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=44 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:43:48.382905 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=30 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:06.922601 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e58a1-5dda-4f58-afb1-7f957fb0bd31 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:06.924815 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-694e58a1-5dda-4f58-afb1-7f957fb0bd31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:07.116887 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e58a1-5dda-4f58-afb1-7f957fb0bd31 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:07.123716 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a613af88-cf9f-4167-be29-fe35a8745470 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:07.124610 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613af88-cf9f-4167-be29-fe35a8745470 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:07.124849 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613af88-cf9f-4167-be29-fe35a8745470 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:07.125127 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613af88-cf9f-4167-be29-fe35a8745470 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:07.127036 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a613af88-cf9f-4167-be29-fe35a8745470 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:07.146862 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a613af88-cf9f-4167-be29-fe35a8745470 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:07.148516 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a613af88-cf9f-4167-be29-fe35a8745470 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:07.161136 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1320/2599] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:44:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:44:07.575874 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58709687-56c4-4b1e-887a-44d25bf24b8f service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:07.737904 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58709687-56c4-4b1e-887a-44d25bf24b8f service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:07.745518 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea413ad-e070-4f14-9b4a-cabf300ee6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:07.746645 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea413ad-e070-4f14-9b4a-cabf300ee6c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:07.747024 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea413ad-e070-4f14-9b4a-cabf300ee6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:07.747320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea413ad-e070-4f14-9b4a-cabf300ee6c3 admin admin] PATH_INFO: `/v3/users/25e86dd7ac714324a6ac8559be408dc7` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:07.749313 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea413ad-e070-4f14-9b4a-cabf300ee6c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25e86dd7ac714324a6ac8559be408dc7)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:07.750887 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea413ad-e070-4f14-9b4a-cabf300ee6c3 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:07.824836 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6ea413ad-e070-4f14-9b4a-cabf300ee6c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25e86dd7ac714324a6ac8559be408dc7'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:07.825600 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6ea413ad-e070-4f14-9b4a-cabf300ee6c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25e86dd7ac714324a6ac8559be408dc7'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:07.841925 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6ea413ad-e070-4f14-9b4a-cabf300ee6c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25e86dd7ac714324a6ac8559be408dc7'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:07.854850 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6ea413ad-e070-4f14-9b4a-cabf300ee6c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25e86dd7ac714324a6ac8559be408dc7'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:07.867639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6ea413ad-e070-4f14-9b4a-cabf300ee6c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25e86dd7ac714324a6ac8559be408dc7'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:07.875431 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1280/2600] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:44:07 2022] DELETE /identity/v3/users/25e86dd7ac714324a6ac8559be408dc7 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:07.880275 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71565e1-baca-45ac-be7f-4bd8fa630498 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:07.880564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71565e1-baca-45ac-be7f-4bd8fa630498 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:07.880885 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71565e1-baca-45ac-be7f-4bd8fa630498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:07.938597 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e71565e1-baca-45ac-be7f-4bd8fa630498 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:44:07.940077 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e71565e1-baca-45ac-be7f-4bd8fa630498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:08.048892 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1321/2601] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:44:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:44:08.058654 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea413ad-e070-4f14-9b4a-cabf300ee6c3 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:08.060182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6ea413ad-e070-4f14-9b4a-cabf300ee6c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:08.124074 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:08.169784 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea413ad-e070-4f14-9b4a-cabf300ee6c3 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:08.176083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff97526-d74a-4825-99b6-64178cc51047 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:08.176994 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff97526-d74a-4825-99b6-64178cc51047 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:08.177753 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff97526-d74a-4825-99b6-64178cc51047 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:08.177753 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff97526-d74a-4825-99b6-64178cc51047 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:08.188832 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff97526-d74a-4825-99b6-64178cc51047 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:08.191150 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff97526-d74a-4825-99b6-64178cc51047 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:08.204478 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1281/2602] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:44:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:44:08.559082 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71565e1-baca-45ac-be7f-4bd8fa630498 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:08.561660 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e71565e1-baca-45ac-be7f-4bd8fa630498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:08.580438 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71565e1-baca-45ac-be7f-4bd8fa630498 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:08.585490 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d482ef-decb-42aa-a5a9-5521311a3443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:08.586426 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d482ef-decb-42aa-a5a9-5521311a3443 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:08.586719 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d482ef-decb-42aa-a5a9-5521311a3443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:08.586974 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d482ef-decb-42aa-a5a9-5521311a3443 admin admin] PATH_INFO: `/v3/projects/9303188058ce403586f38325a1deaea0` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:08.588249 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d482ef-decb-42aa-a5a9-5521311a3443 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9303188058ce403586f38325a1deaea0)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:08.590295 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d482ef-decb-42aa-a5a9-5521311a3443 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:08.611332 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:08.653341 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-05d482ef-decb-42aa-a5a9-5521311a3443 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9303188058ce403586f38325a1deaea0', 'request_id': 'req-05d482ef-decb-42aa-a5a9-5521311a3443'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:08.662407 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-05d482ef-decb-42aa-a5a9-5521311a3443 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9303188058ce403586f38325a1deaea0', 'request_id': 'req-05d482ef-decb-42aa-a5a9-5521311a3443'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:08.676754 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-05d482ef-decb-42aa-a5a9-5521311a3443 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9303188058ce403586f38325a1deaea0', 'request_id': 'req-05d482ef-decb-42aa-a5a9-5521311a3443'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:08.685816 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-05d482ef-decb-42aa-a5a9-5521311a3443 admin admin] The token cache is being invalidate because project 9303188058ce403586f38325a1deaea0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:44:08.686532 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-05d482ef-decb-42aa-a5a9-5521311a3443 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:08.687301 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-05d482ef-decb-42aa-a5a9-5521311a3443 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:08.689042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1322/2603] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:44:08 2022] DELETE /identity/v3/projects/9303188058ce403586f38325a1deaea0 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:08.693130 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff97526-d74a-4825-99b6-64178cc51047 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:08.703610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff97526-d74a-4825-99b6-64178cc51047 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:08.764222 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f46112-1414-42c0-a508-1eaf0d4186ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:08.765215 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f46112-1414-42c0-a508-1eaf0d4186ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:08.765501 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f46112-1414-42c0-a508-1eaf0d4186ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:08.765780 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f46112-1414-42c0-a508-1eaf0d4186ee admin admin] PATH_INFO: `/v3/users/668cc44b4bbc46b5bc24793f8d55e37d` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:08.767088 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f46112-1414-42c0-a508-1eaf0d4186ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=668cc44b4bbc46b5bc24793f8d55e37d)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:08.769127 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f46112-1414-42c0-a508-1eaf0d4186ee admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:08.834856 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a2f46112-1414-42c0-a508-1eaf0d4186ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '668cc44b4bbc46b5bc24793f8d55e37d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:08.835940 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a2f46112-1414-42c0-a508-1eaf0d4186ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '668cc44b4bbc46b5bc24793f8d55e37d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:08.862876 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a2f46112-1414-42c0-a508-1eaf0d4186ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '668cc44b4bbc46b5bc24793f8d55e37d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:08.874359 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a2f46112-1414-42c0-a508-1eaf0d4186ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '668cc44b4bbc46b5bc24793f8d55e37d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:08.884715 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a2f46112-1414-42c0-a508-1eaf0d4186ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '668cc44b4bbc46b5bc24793f8d55e37d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:08.892343 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1282/2604] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:44:08 2022] DELETE /identity/v3/users/668cc44b4bbc46b5bc24793f8d55e37d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:09.252916 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d482ef-decb-42aa-a5a9-5521311a3443 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:09.254839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-05d482ef-decb-42aa-a5a9-5521311a3443 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:09.279210 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d482ef-decb-42aa-a5a9-5521311a3443 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:09.346754 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd8da16-fb3f-42cc-be31-471ce6c8129c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:09.347654 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8da16-fb3f-42cc-be31-471ce6c8129c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:09.347925 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8da16-fb3f-42cc-be31-471ce6c8129c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:09.348201 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8da16-fb3f-42cc-be31-471ce6c8129c admin admin] PATH_INFO: `/v3/projects/932f862d022d4ee8961662e071e806fb` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:09.349649 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd8da16-fb3f-42cc-be31-471ce6c8129c admin admin] RBAC: Authorizing `identity:delete_project(project_id=932f862d022d4ee8961662e071e806fb)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:09.351830 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd8da16-fb3f-42cc-be31-471ce6c8129c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:09.409049 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6cd8da16-fb3f-42cc-be31-471ce6c8129c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '932f862d022d4ee8961662e071e806fb', 'request_id': 'req-6cd8da16-fb3f-42cc-be31-471ce6c8129c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:09.415385 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6cd8da16-fb3f-42cc-be31-471ce6c8129c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '932f862d022d4ee8961662e071e806fb', 'request_id': 'req-6cd8da16-fb3f-42cc-be31-471ce6c8129c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:09.421980 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6cd8da16-fb3f-42cc-be31-471ce6c8129c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '932f862d022d4ee8961662e071e806fb', 'request_id': 'req-6cd8da16-fb3f-42cc-be31-471ce6c8129c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:09.436555 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6cd8da16-fb3f-42cc-be31-471ce6c8129c admin admin] The token cache is being invalidate because project 932f862d022d4ee8961662e071e806fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:44:09.436848 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6cd8da16-fb3f-42cc-be31-471ce6c8129c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:09.437569 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6cd8da16-fb3f-42cc-be31-471ce6c8129c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:09.439256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1323/2605] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:44:09 2022] DELETE /identity/v3/projects/932f862d022d4ee8961662e071e806fb => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:09.445147 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857f65a-dd88-4572-a25a-30dad5dace39 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:09.445438 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857f65a-dd88-4572-a25a-30dad5dace39 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:09.445714 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857f65a-dd88-4572-a25a-30dad5dace39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:09.495916 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-3857f65a-dd88-4572-a25a-30dad5dace39 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:44:09.497144 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3857f65a-dd88-4572-a25a-30dad5dace39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:09.509913 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:09.573327 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1283/2606] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:44:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:44:09.581934 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916a9b7-1f33-4829-b8f4-c155fcf38257 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:09.582233 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916a9b7-1f33-4829-b8f4-c155fcf38257 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:09.582534 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916a9b7-1f33-4829-b8f4-c155fcf38257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:09.602129 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9916a9b7-1f33-4829-b8f4-c155fcf38257 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:44:09.603390 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9916a9b7-1f33-4829-b8f4-c155fcf38257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:09.611643 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1324/2607] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:44:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:44:09.616062 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857f65a-dd88-4572-a25a-30dad5dace39 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:09.626427 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857f65a-dd88-4572-a25a-30dad5dace39 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:09.632249 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd92076-1ecc-43ee-b04b-f7f9abe036fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:09.632926 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd92076-1ecc-43ee-b04b-f7f9abe036fb admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:09.633210 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd92076-1ecc-43ee-b04b-f7f9abe036fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:09.633485 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd92076-1ecc-43ee-b04b-f7f9abe036fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:09.634224 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd92076-1ecc-43ee-b04b-f7f9abe036fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:44:09.634573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd92076-1ecc-43ee-b04b-f7f9abe036fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:09.636191 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd92076-1ecc-43ee-b04b-f7f9abe036fb admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:09.648463 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1284/2608] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:44:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:44:09.653001 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9916a9b7-1f33-4829-b8f4-c155fcf38257 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:09.662911 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9916a9b7-1f33-4829-b8f4-c155fcf38257 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:09.667738 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5498f3-4b35-4318-8fdf-43ad919d3d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:09.668413 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5498f3-4b35-4318-8fdf-43ad919d3d53 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:09.668694 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5498f3-4b35-4318-8fdf-43ad919d3d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:09.668954 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5498f3-4b35-4318-8fdf-43ad919d3d53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:09.669662 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5498f3-4b35-4318-8fdf-43ad919d3d53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:09.672183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5498f3-4b35-4318-8fdf-43ad919d3d53 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:09.700386 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1325/2609] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:44:09 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:44:09.704996 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd92076-1ecc-43ee-b04b-f7f9abe036fb admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:09.716174 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd92076-1ecc-43ee-b04b-f7f9abe036fb admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:09.788322 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c68d7-6f74-44b5-81f5-7103465faf60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:09.788322 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c68d7-6f74-44b5-81f5-7103465faf60 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:09.788533 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c68d7-6f74-44b5-81f5-7103465faf60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:09.791358 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c68d7-6f74-44b5-81f5-7103465faf60 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:09.791358 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997c68d7-6f74-44b5-81f5-7103465faf60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:09.792439 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997c68d7-6f74-44b5-81f5-7103465faf60 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:09.818885 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1285/2610] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:44:09 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:44:09.824741 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5498f3-4b35-4318-8fdf-43ad919d3d53 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:09.832494 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:09.837863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5498f3-4b35-4318-8fdf-43ad919d3d53 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:09.851951 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f2aa0-fc70-4500-9dfb-57022766482c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:09.858927 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f2aa0-fc70-4500-9dfb-57022766482c admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:09.859211 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f2aa0-fc70-4500-9dfb-57022766482c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:09.859652 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f2aa0-fc70-4500-9dfb-57022766482c admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:09.860284 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356f2aa0-fc70-4500-9dfb-57022766482c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:09.861994 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356f2aa0-fc70-4500-9dfb-57022766482c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:09.874980 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1326/2611] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:44:09.879027 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c68d7-6f74-44b5-81f5-7103465faf60 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:09.890596 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c68d7-6f74-44b5-81f5-7103465faf60 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:09.895661 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e0005-0074-45cd-a072-56b2c8df5a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:09.896436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e0005-0074-45cd-a072-56b2c8df5a3f admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:09.896923 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e0005-0074-45cd-a072-56b2c8df5a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:09.897199 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e0005-0074-45cd-a072-56b2c8df5a3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:09.897938 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e0005-0074-45cd-a072-56b2c8df5a3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:09.899668 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e0005-0074-45cd-a072-56b2c8df5a3f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:09.914077 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1286/2612] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:44:09.916282 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f2aa0-fc70-4500-9dfb-57022766482c admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:09.928866 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f2aa0-fc70-4500-9dfb-57022766482c admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:09.934665 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d072ad9-5864-45ef-9a9b-3e1e3ca44bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:09.935621 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d072ad9-5864-45ef-9a9b-3e1e3ca44bfc admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:09.935906 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d072ad9-5864-45ef-9a9b-3e1e3ca44bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:09.936362 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d072ad9-5864-45ef-9a9b-3e1e3ca44bfc admin admin] PATH_INFO: `/v3/projects/5927632ec4944b16a27cfeeddcfc10af/users/305d6f41848a4dc6b4bbb762875fa097/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:09.953358 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d072ad9-5864-45ef-9a9b-3e1e3ca44bfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5927632ec4944b16a27cfeeddcfc10af, user_id=305d6f41848a4dc6b4bbb762875fa097, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:09.956104 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d072ad9-5864-45ef-9a9b-3e1e3ca44bfc admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:09.972397 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1327/2613] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:44:09 2022] PUT /identity/v3/projects/5927632ec4944b16a27cfeeddcfc10af/users/305d6f41848a4dc6b4bbb762875fa097/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:09.984221 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69782bb3-4ccb-438e-b874-0792a524306e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:09.984507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69782bb3-4ccb-438e-b874-0792a524306e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:09.984783 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69782bb3-4ccb-438e-b874-0792a524306e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:10.015043 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-69782bb3-4ccb-438e-b874-0792a524306e None None] MFA Rules not processed for user `305d6f41848a4dc6b4bbb762875fa097`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:44:10.016276 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-69782bb3-4ccb-438e-b874-0792a524306e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:10.086257 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1287/2614] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:44:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:44:10.091079 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d072ad9-5864-45ef-9a9b-3e1e3ca44bfc admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:10.156314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d072ad9-5864-45ef-9a9b-3e1e3ca44bfc admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:10.161534 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f44b18-71c5-4265-9e21-4ad584fb27f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:10.162250 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f44b18-71c5-4265-9e21-4ad584fb27f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:10.162565 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f44b18-71c5-4265-9e21-4ad584fb27f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:10.162836 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f44b18-71c5-4265-9e21-4ad584fb27f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:10.163544 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f44b18-71c5-4265-9e21-4ad584fb27f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:10.165540 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f44b18-71c5-4265-9e21-4ad584fb27f1 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:10.194980 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1328/2615] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:44:10 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:44:10.201428 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69782bb3-4ccb-438e-b874-0792a524306e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:10.218115 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69782bb3-4ccb-438e-b874-0792a524306e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:10.228198 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:10.282486 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1881a73-bd05-4de9-8283-8c2cde4b75fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:10.283185 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1881a73-bd05-4de9-8283-8c2cde4b75fc admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:10.283470 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1881a73-bd05-4de9-8283-8c2cde4b75fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:10.283748 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1881a73-bd05-4de9-8283-8c2cde4b75fc admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:10.284508 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1881a73-bd05-4de9-8283-8c2cde4b75fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:10.286512 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1881a73-bd05-4de9-8283-8c2cde4b75fc admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:10.312129 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1288/2616] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:44:10 2022] POST /identity/v3/users => generated 298 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:44:10.316247 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f44b18-71c5-4265-9e21-4ad584fb27f1 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:10.328254 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f44b18-71c5-4265-9e21-4ad584fb27f1 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:10.333545 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83322b26-7aa4-49c1-a7b4-bd5d5306c288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:10.334263 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83322b26-7aa4-49c1-a7b4-bd5d5306c288 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:10.334543 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83322b26-7aa4-49c1-a7b4-bd5d5306c288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:10.334838 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83322b26-7aa4-49c1-a7b4-bd5d5306c288 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:10.335601 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83322b26-7aa4-49c1-a7b4-bd5d5306c288 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:10.337329 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83322b26-7aa4-49c1-a7b4-bd5d5306c288 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:10.349933 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1329/2617] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:44:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:44:10.355913 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1881a73-bd05-4de9-8283-8c2cde4b75fc admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:10.378784 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1881a73-bd05-4de9-8283-8c2cde4b75fc admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:10.384383 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7193d5-8ae8-4c86-bb93-472ca299fa33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:10.385481 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7193d5-8ae8-4c86-bb93-472ca299fa33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:10.385737 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7193d5-8ae8-4c86-bb93-472ca299fa33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:10.386040 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7193d5-8ae8-4c86-bb93-472ca299fa33 admin admin] PATH_INFO: `/v3/projects/25f2ae91a567418393c328c5042b1444/users/96f6f88e0e084a59ada9a8aba3dcbb13/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:10.407870 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7193d5-8ae8-4c86-bb93-472ca299fa33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25f2ae91a567418393c328c5042b1444, user_id=96f6f88e0e084a59ada9a8aba3dcbb13, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:10.411880 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7193d5-8ae8-4c86-bb93-472ca299fa33 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:10.428212 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1289/2618] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:44:10 2022] PUT /identity/v3/projects/25f2ae91a567418393c328c5042b1444/users/96f6f88e0e084a59ada9a8aba3dcbb13/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:10.432409 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83322b26-7aa4-49c1-a7b4-bd5d5306c288 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:10.447930 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83322b26-7aa4-49c1-a7b4-bd5d5306c288 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:10.510402 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375db1ba-35aa-494c-aca7-4b7f902c1286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:10.511208 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375db1ba-35aa-494c-aca7-4b7f902c1286 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:10.511393 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375db1ba-35aa-494c-aca7-4b7f902c1286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:10.511668 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375db1ba-35aa-494c-aca7-4b7f902c1286 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:10.512732 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375db1ba-35aa-494c-aca7-4b7f902c1286 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:10.514070 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375db1ba-35aa-494c-aca7-4b7f902c1286 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:10.525116 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1330/2619] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:44:10 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:44:10.531827 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7193d5-8ae8-4c86-bb93-472ca299fa33 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:10.546832 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7193d5-8ae8-4c86-bb93-472ca299fa33 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:10.554817 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248c893-d097-4dab-94df-33226960e5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:10.556153 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248c893-d097-4dab-94df-33226960e5b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:10.556541 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248c893-d097-4dab-94df-33226960e5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:10.556949 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248c893-d097-4dab-94df-33226960e5b3 admin admin] PATH_INFO: `/v3/projects/25f2ae91a567418393c328c5042b1444/users/96f6f88e0e084a59ada9a8aba3dcbb13/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:10.560262 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7248c893-d097-4dab-94df-33226960e5b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25f2ae91a567418393c328c5042b1444, user_id=96f6f88e0e084a59ada9a8aba3dcbb13, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:10.563293 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7248c893-d097-4dab-94df-33226960e5b3 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:10.574958 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1290/2620] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:44:10 2022] PUT /identity/v3/projects/25f2ae91a567418393c328c5042b1444/users/96f6f88e0e084a59ada9a8aba3dcbb13/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:10.590660 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332af61f-d7c2-4776-9673-53639ae20f54 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:10.591144 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332af61f-d7c2-4776-9673-53639ae20f54 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:10.591609 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332af61f-d7c2-4776-9673-53639ae20f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:10.620922 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-332af61f-d7c2-4776-9673-53639ae20f54 None None] MFA Rules not processed for user `96f6f88e0e084a59ada9a8aba3dcbb13`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:44:10.623652 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-332af61f-d7c2-4776-9673-53639ae20f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:10.654376 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:10.697862 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1331/2621] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:44:10 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:44:10.715578 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248c893-d097-4dab-94df-33226960e5b3 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:10.717861 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7248c893-d097-4dab-94df-33226960e5b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:10.780280 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248c893-d097-4dab-94df-33226960e5b3 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:10.784241 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da0866-e954-4c79-9389-c5d363445ecb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:10.784893 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da0866-e954-4c79-9389-c5d363445ecb service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:10.785167 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da0866-e954-4c79-9389-c5d363445ecb service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:10.785436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da0866-e954-4c79-9389-c5d363445ecb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:10.794806 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da0866-e954-4c79-9389-c5d363445ecb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:10.796377 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da0866-e954-4c79-9389-c5d363445ecb service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:10.806191 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1291/2622] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:44:10 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:44:10.918211 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332af61f-d7c2-4776-9673-53639ae20f54 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:10.929384 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332af61f-d7c2-4776-9673-53639ae20f54 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:10.933364 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf374b-bbb7-4523-972a-ea641eaf155b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:10.934040 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf374b-bbb7-4523-972a-ea641eaf155b service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:10.934386 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf374b-bbb7-4523-972a-ea641eaf155b service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:10.934650 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf374b-bbb7-4523-972a-ea641eaf155b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:11.004145 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf374b-bbb7-4523-972a-ea641eaf155b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:11.005610 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf374b-bbb7-4523-972a-ea641eaf155b service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:11.017354 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1332/2623] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:44:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:44:11.033042 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da0866-e954-4c79-9389-c5d363445ecb service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:11.034539 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-99da0866-e954-4c79-9389-c5d363445ecb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:11.105170 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da0866-e954-4c79-9389-c5d363445ecb service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:11.111211 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf37f005-7623-463e-9731-d6bd925597a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:11.111791 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37f005-7623-463e-9731-d6bd925597a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:11.112076 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37f005-7623-463e-9731-d6bd925597a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:11.112495 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37f005-7623-463e-9731-d6bd925597a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:11.114832 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-bf37f005-7623-463e-9731-d6bd925597a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:11.147110 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:11.196364 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf37f005-7623-463e-9731-d6bd925597a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:11.198356 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf37f005-7623-463e-9731-d6bd925597a1 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:11.209915 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1292/2624] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:44:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:44:13.667019 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf374b-bbb7-4523-972a-ea641eaf155b service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:13.678710 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf374b-bbb7-4523-972a-ea641eaf155b service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:13.686055 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e5e0ba-bc45-4641-999f-4af7f9df94bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:13.687221 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e5e0ba-bc45-4641-999f-4af7f9df94bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:13.687505 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e5e0ba-bc45-4641-999f-4af7f9df94bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:13.687953 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e5e0ba-bc45-4641-999f-4af7f9df94bc admin admin] PATH_INFO: `/v3/users/305d6f41848a4dc6b4bbb762875fa097` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:13.689303 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e5e0ba-bc45-4641-999f-4af7f9df94bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=305d6f41848a4dc6b4bbb762875fa097)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:13.691400 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e5e0ba-bc45-4641-999f-4af7f9df94bc admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:13.740816 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-92e5e0ba-bc45-4641-999f-4af7f9df94bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '305d6f41848a4dc6b4bbb762875fa097'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:13.752995 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-92e5e0ba-bc45-4641-999f-4af7f9df94bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '305d6f41848a4dc6b4bbb762875fa097'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:13.753697 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-92e5e0ba-bc45-4641-999f-4af7f9df94bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '305d6f41848a4dc6b4bbb762875fa097'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:13.754328 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-92e5e0ba-bc45-4641-999f-4af7f9df94bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '305d6f41848a4dc6b4bbb762875fa097'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:13.761202 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-92e5e0ba-bc45-4641-999f-4af7f9df94bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '305d6f41848a4dc6b4bbb762875fa097'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:13.775422 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1333/2625] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:44:13 2022] DELETE /identity/v3/users/305d6f41848a4dc6b4bbb762875fa097 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:13.780416 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf2f76a-7242-44d4-8b67-681a03e77eab None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:13.780945 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf2f76a-7242-44d4-8b67-681a03e77eab None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:13.781142 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf2f76a-7242-44d4-8b67-681a03e77eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:13.799934 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-3bf2f76a-7242-44d4-8b67-681a03e77eab None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:44:13.801159 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3bf2f76a-7242-44d4-8b67-681a03e77eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:13.877010 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1293/2626] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:44:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:44:13.886958 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e5e0ba-bc45-4641-999f-4af7f9df94bc admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:13.888653 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-92e5e0ba-bc45-4641-999f-4af7f9df94bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:13.931055 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:13.959017 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e5e0ba-bc45-4641-999f-4af7f9df94bc admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:13.964505 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93284885-84f8-49e1-b34e-3c8c392987ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:13.965524 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93284885-84f8-49e1-b34e-3c8c392987ee service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:13.965990 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93284885-84f8-49e1-b34e-3c8c392987ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:13.966810 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93284885-84f8-49e1-b34e-3c8c392987ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:13.983132 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93284885-84f8-49e1-b34e-3c8c392987ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:13.986152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93284885-84f8-49e1-b34e-3c8c392987ee service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:14.003481 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1334/2627] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:44:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:44:14.499051 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf2f76a-7242-44d4-8b67-681a03e77eab None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:14.500257 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3bf2f76a-7242-44d4-8b67-681a03e77eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:14.539960 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf2f76a-7242-44d4-8b67-681a03e77eab None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:14.561578 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5192405-936f-4c9b-a1b6-5be1c4f4f81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:14.562635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5192405-936f-4c9b-a1b6-5be1c4f4f81d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:14.563016 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5192405-936f-4c9b-a1b6-5be1c4f4f81d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:14.563902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5192405-936f-4c9b-a1b6-5be1c4f4f81d admin admin] PATH_INFO: `/v3/projects/5927632ec4944b16a27cfeeddcfc10af` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:14.566027 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5192405-936f-4c9b-a1b6-5be1c4f4f81d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5927632ec4944b16a27cfeeddcfc10af)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:14.568189 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5192405-936f-4c9b-a1b6-5be1c4f4f81d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:14.667743 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e5192405-936f-4c9b-a1b6-5be1c4f4f81d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5927632ec4944b16a27cfeeddcfc10af', 'request_id': 'req-e5192405-936f-4c9b-a1b6-5be1c4f4f81d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:14.677711 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e5192405-936f-4c9b-a1b6-5be1c4f4f81d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5927632ec4944b16a27cfeeddcfc10af', 'request_id': 'req-e5192405-936f-4c9b-a1b6-5be1c4f4f81d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:14.690907 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e5192405-936f-4c9b-a1b6-5be1c4f4f81d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5927632ec4944b16a27cfeeddcfc10af', 'request_id': 'req-e5192405-936f-4c9b-a1b6-5be1c4f4f81d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:14.705298 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e5192405-936f-4c9b-a1b6-5be1c4f4f81d admin admin] The token cache is being invalidate because project 5927632ec4944b16a27cfeeddcfc10af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:44:14.705660 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e5192405-936f-4c9b-a1b6-5be1c4f4f81d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:14.706440 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e5192405-936f-4c9b-a1b6-5be1c4f4f81d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:14.711926 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1294/2628] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:44:14 2022] DELETE /identity/v3/projects/5927632ec4944b16a27cfeeddcfc10af => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:14.713474 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93284885-84f8-49e1-b34e-3c8c392987ee service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:14.731791 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93284885-84f8-49e1-b34e-3c8c392987ee service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:14.859474 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68339089-14a4-4859-b469-e6c845bbe1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:14.862137 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68339089-14a4-4859-b469-e6c845bbe1fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:14.862608 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68339089-14a4-4859-b469-e6c845bbe1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:14.863049 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68339089-14a4-4859-b469-e6c845bbe1fb admin admin] PATH_INFO: `/v3/users/96f6f88e0e084a59ada9a8aba3dcbb13` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:14.864510 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68339089-14a4-4859-b469-e6c845bbe1fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=96f6f88e0e084a59ada9a8aba3dcbb13)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:14.868562 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68339089-14a4-4859-b469-e6c845bbe1fb admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:14.911698 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5192405-936f-4c9b-a1b6-5be1c4f4f81d admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:14.913834 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e5192405-936f-4c9b-a1b6-5be1c4f4f81d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:14.943539 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-68339089-14a4-4859-b469-e6c845bbe1fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96f6f88e0e084a59ada9a8aba3dcbb13'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:14.948878 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5192405-936f-4c9b-a1b6-5be1c4f4f81d admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:14.954767 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832e0436-13f1-411b-960d-a3b82cf87fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:14.955726 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832e0436-13f1-411b-960d-a3b82cf87fe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:14.956010 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832e0436-13f1-411b-960d-a3b82cf87fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:14.956285 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832e0436-13f1-411b-960d-a3b82cf87fe6 admin admin] PATH_INFO: `/v3/users/f47e7f746a014cb0a35390e7e0ce435d` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:14.956738 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-68339089-14a4-4859-b469-e6c845bbe1fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96f6f88e0e084a59ada9a8aba3dcbb13'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:14.957431 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-68339089-14a4-4859-b469-e6c845bbe1fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96f6f88e0e084a59ada9a8aba3dcbb13'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:14.957539 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832e0436-13f1-411b-960d-a3b82cf87fe6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f47e7f746a014cb0a35390e7e0ce435d)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:14.958074 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-68339089-14a4-4859-b469-e6c845bbe1fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96f6f88e0e084a59ada9a8aba3dcbb13'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:14.959745 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832e0436-13f1-411b-960d-a3b82cf87fe6 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:14.965690 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-68339089-14a4-4859-b469-e6c845bbe1fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96f6f88e0e084a59ada9a8aba3dcbb13'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:14.980981 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1335/2629] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:44:14 2022] DELETE /identity/v3/users/96f6f88e0e084a59ada9a8aba3dcbb13 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:15.000039 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:15.011920 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-832e0436-13f1-411b-960d-a3b82cf87fe6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f47e7f746a014cb0a35390e7e0ce435d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:15.012931 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-832e0436-13f1-411b-960d-a3b82cf87fe6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f47e7f746a014cb0a35390e7e0ce435d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:15.029160 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-832e0436-13f1-411b-960d-a3b82cf87fe6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f47e7f746a014cb0a35390e7e0ce435d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:15.046943 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-832e0436-13f1-411b-960d-a3b82cf87fe6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f47e7f746a014cb0a35390e7e0ce435d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:15.056609 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-832e0436-13f1-411b-960d-a3b82cf87fe6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f47e7f746a014cb0a35390e7e0ce435d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:15.064717 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1295/2630] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:44:14 2022] DELETE /identity/v3/users/f47e7f746a014cb0a35390e7e0ce435d => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:44:15.274273 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68339089-14a4-4859-b469-e6c845bbe1fb admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:15.277716 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-68339089-14a4-4859-b469-e6c845bbe1fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:15.291515 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832e0436-13f1-411b-960d-a3b82cf87fe6 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:15.293831 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-832e0436-13f1-411b-960d-a3b82cf87fe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:15.357646 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:15.362185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68339089-14a4-4859-b469-e6c845bbe1fb admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:15.370511 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832e0436-13f1-411b-960d-a3b82cf87fe6 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:15.375434 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15db454-f4ae-408a-9260-eae93ed34548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:15.376216 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab6270-6752-4ac5-9143-834e6e46e962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:15.376598 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15db454-f4ae-408a-9260-eae93ed34548 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:15.377122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15db454-f4ae-408a-9260-eae93ed34548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:15.377239 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab6270-6752-4ac5-9143-834e6e46e962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:15.377461 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab6270-6752-4ac5-9143-834e6e46e962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:15.377584 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15db454-f4ae-408a-9260-eae93ed34548 admin admin] PATH_INFO: `/v3/projects/92434c51e27a405aaa89a70a660f6d91` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:15.377798 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab6270-6752-4ac5-9143-834e6e46e962 admin admin] PATH_INFO: `/v3/projects/25f2ae91a567418393c328c5042b1444` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:15.378917 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15db454-f4ae-408a-9260-eae93ed34548 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92434c51e27a405aaa89a70a660f6d91)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:15.379044 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ab6270-6752-4ac5-9143-834e6e46e962 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25f2ae91a567418393c328c5042b1444)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:15.380905 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ab6270-6752-4ac5-9143-834e6e46e962 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:15.381033 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15db454-f4ae-408a-9260-eae93ed34548 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:15.432459 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e15db454-f4ae-408a-9260-eae93ed34548 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92434c51e27a405aaa89a70a660f6d91', 'request_id': 'req-e15db454-f4ae-408a-9260-eae93ed34548'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:15.433453 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-08ab6270-6752-4ac5-9143-834e6e46e962 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25f2ae91a567418393c328c5042b1444', 'request_id': 'req-08ab6270-6752-4ac5-9143-834e6e46e962'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:15.439198 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e15db454-f4ae-408a-9260-eae93ed34548 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92434c51e27a405aaa89a70a660f6d91', 'request_id': 'req-e15db454-f4ae-408a-9260-eae93ed34548'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:15.439989 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-08ab6270-6752-4ac5-9143-834e6e46e962 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25f2ae91a567418393c328c5042b1444', 'request_id': 'req-08ab6270-6752-4ac5-9143-834e6e46e962'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:15.445178 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e15db454-f4ae-408a-9260-eae93ed34548 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92434c51e27a405aaa89a70a660f6d91', 'request_id': 'req-e15db454-f4ae-408a-9260-eae93ed34548'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:15.449877 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-08ab6270-6752-4ac5-9143-834e6e46e962 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25f2ae91a567418393c328c5042b1444', 'request_id': 'req-08ab6270-6752-4ac5-9143-834e6e46e962'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:15.460053 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e15db454-f4ae-408a-9260-eae93ed34548 admin admin] The token cache is being invalidate because project 92434c51e27a405aaa89a70a660f6d91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:44:15.460470 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e15db454-f4ae-408a-9260-eae93ed34548 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:15.461169 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e15db454-f4ae-408a-9260-eae93ed34548 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:15.461592 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-08ab6270-6752-4ac5-9143-834e6e46e962 admin admin] The token cache is being invalidate because project 25f2ae91a567418393c328c5042b1444 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:44:15.461883 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-08ab6270-6752-4ac5-9143-834e6e46e962 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:15.462538 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-08ab6270-6752-4ac5-9143-834e6e46e962 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:15.463157 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1336/2631] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:44:15 2022] DELETE /identity/v3/projects/92434c51e27a405aaa89a70a660f6d91 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:15.464107 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1296/2632] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:44:15 2022] DELETE /identity/v3/projects/25f2ae91a567418393c328c5042b1444 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:15.468244 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64059ed7-825b-428e-98f1-1d3809a8d638 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:15.468539 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64059ed7-825b-428e-98f1-1d3809a8d638 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:15.468823 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64059ed7-825b-428e-98f1-1d3809a8d638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:15.468992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652866ed-471a-4741-9afd-072127ec1783 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:15.469310 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652866ed-471a-4741-9afd-072127ec1783 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:15.469587 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652866ed-471a-4741-9afd-072127ec1783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:15.487537 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-64059ed7-825b-428e-98f1-1d3809a8d638 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:44:15.488026 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-652866ed-471a-4741-9afd-072127ec1783 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:44:15.488760 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-64059ed7-825b-428e-98f1-1d3809a8d638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:15.489255 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-652866ed-471a-4741-9afd-072127ec1783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:15.527961 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:15.550645 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1337/2633] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:44:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:44:15.550879 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1297/2634] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:44:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:44:15.558482 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb33d7-8c2d-4ec6-84c9-eb37b85a099f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:15.558775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb33d7-8c2d-4ec6-84c9-eb37b85a099f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:15.559052 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb33d7-8c2d-4ec6-84c9-eb37b85a099f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:15.559340 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45319e9-dc8b-4b66-bb60-b5e818ffe57f None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:15.559633 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45319e9-dc8b-4b66-bb60-b5e818ffe57f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:15.559909 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45319e9-dc8b-4b66-bb60-b5e818ffe57f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:15.580077 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-fbcb33d7-8c2d-4ec6-84c9-eb37b85a099f None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:44:15.580823 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-d45319e9-dc8b-4b66-bb60-b5e818ffe57f None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:44:15.581301 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fbcb33d7-8c2d-4ec6-84c9-eb37b85a099f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:15.581986 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d45319e9-dc8b-4b66-bb60-b5e818ffe57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:15.589347 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1338/2635] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:44:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:44:15.590054 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1298/2636] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:44:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:44:15.593968 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb33d7-8c2d-4ec6-84c9-eb37b85a099f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:15.594134 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45319e9-dc8b-4b66-bb60-b5e818ffe57f None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:15.604055 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb33d7-8c2d-4ec6-84c9-eb37b85a099f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:15.604380 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45319e9-dc8b-4b66-bb60-b5e818ffe57f None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:15.609233 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a79e8-e2a5-4316-80ad-ff1430efca58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:15.609905 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a79e8-e2a5-4316-80ad-ff1430efca58 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:15.610187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a79e8-e2a5-4316-80ad-ff1430efca58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:15.610483 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a79e8-e2a5-4316-80ad-ff1430efca58 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:15.611219 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a79e8-e2a5-4316-80ad-ff1430efca58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:44:15.611524 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a79e8-e2a5-4316-80ad-ff1430efca58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:15.613150 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a79e8-e2a5-4316-80ad-ff1430efca58 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:15.613821 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b1177-6d83-41f2-a021-f2db04ae7836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:15.614590 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b1177-6d83-41f2-a021-f2db04ae7836 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:15.614944 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b1177-6d83-41f2-a021-f2db04ae7836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:15.615224 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b1177-6d83-41f2-a021-f2db04ae7836 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:15.615955 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b1177-6d83-41f2-a021-f2db04ae7836 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:44:15.616257 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b1177-6d83-41f2-a021-f2db04ae7836 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:15.618294 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b1177-6d83-41f2-a021-f2db04ae7836 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:15.626877 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1339/2637] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:44:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:44:15.631602 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a79e8-e2a5-4316-80ad-ff1430efca58 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:15.633442 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1299/2638] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:44:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 21:44:15.637566 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b1177-6d83-41f2-a021-f2db04ae7836 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:15.644679 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a79e8-e2a5-4316-80ad-ff1430efca58 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:15.647976 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b1177-6d83-41f2-a021-f2db04ae7836 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:15.649841 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e8ea7-6abb-4321-9bec-839f92184a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:15.650687 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e8ea7-6abb-4321-9bec-839f92184a7c admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:15.651071 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e8ea7-6abb-4321-9bec-839f92184a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:15.651352 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e8ea7-6abb-4321-9bec-839f92184a7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:15.652077 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02e8ea7-6abb-4321-9bec-839f92184a7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:15.652802 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c427f-ae2d-4c2e-ad4b-d7132e7c4aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:15.653461 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c427f-ae2d-4c2e-ad4b-d7132e7c4aee admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:15.653742 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c427f-ae2d-4c2e-ad4b-d7132e7c4aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:15.653988 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02e8ea7-6abb-4321-9bec-839f92184a7c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:15.654086 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c427f-ae2d-4c2e-ad4b-d7132e7c4aee admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:15.654760 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c427f-ae2d-4c2e-ad4b-d7132e7c4aee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:15.656677 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c427f-ae2d-4c2e-ad4b-d7132e7c4aee admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:15.682758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1340/2639] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:44:15 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:44:15.684667 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1300/2640] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:44:15 2022] POST /identity/v3/projects => generated 409 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:44:15.687070 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e8ea7-6abb-4321-9bec-839f92184a7c admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:15.690605 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c427f-ae2d-4c2e-ad4b-d7132e7c4aee admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:15.740617 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:15.756731 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e8ea7-6abb-4321-9bec-839f92184a7c admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:15.759588 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c427f-ae2d-4c2e-ad4b-d7132e7c4aee admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:15.762072 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388f280-6fb4-49df-8289-ae58433789cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:15.762808 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388f280-6fb4-49df-8289-ae58433789cd admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:15.763157 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388f280-6fb4-49df-8289-ae58433789cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:15.763384 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388f280-6fb4-49df-8289-ae58433789cd admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:15.764303 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388f280-6fb4-49df-8289-ae58433789cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:15.764422 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a1082b-9e90-481f-ab83-b48086e05258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:15.765085 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1082b-9e90-481f-ab83-b48086e05258 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:15.765375 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1082b-9e90-481f-ab83-b48086e05258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:15.765650 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1082b-9e90-481f-ab83-b48086e05258 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:15.766164 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388f280-6fb4-49df-8289-ae58433789cd admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:15.766374 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a1082b-9e90-481f-ab83-b48086e05258 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:15.768338 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a1082b-9e90-481f-ab83-b48086e05258 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:15.791012 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1341/2641] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:44:15 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:44:15.795078 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1301/2642] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:44:15 2022] POST /identity/v3/users => generated 305 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:44:15.795549 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388f280-6fb4-49df-8289-ae58433789cd admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:15.799311 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a1082b-9e90-481f-ab83-b48086e05258 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:15.806030 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388f280-6fb4-49df-8289-ae58433789cd admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:15.810509 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a1082b-9e90-481f-ab83-b48086e05258 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:15.810894 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c077462-3d36-42a1-b0f5-dbe50222d17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:15.811769 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c077462-3d36-42a1-b0f5-dbe50222d17b admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:15.812005 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c077462-3d36-42a1-b0f5-dbe50222d17b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:15.812281 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c077462-3d36-42a1-b0f5-dbe50222d17b admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:15.813151 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c077462-3d36-42a1-b0f5-dbe50222d17b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:15.814670 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c077462-3d36-42a1-b0f5-dbe50222d17b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:15.815255 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a362e82d-6d26-4dd3-ae58-c172c487b521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:15.815916 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a362e82d-6d26-4dd3-ae58-c172c487b521 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:15.816197 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a362e82d-6d26-4dd3-ae58-c172c487b521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:15.816471 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a362e82d-6d26-4dd3-ae58-c172c487b521 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:15.817223 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a362e82d-6d26-4dd3-ae58-c172c487b521 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:15.818862 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a362e82d-6d26-4dd3-ae58-c172c487b521 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:15.827881 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1342/2643] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:44:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:44:15.830888 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1302/2644] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:44:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 21:44:15.833051 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c077462-3d36-42a1-b0f5-dbe50222d17b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:15.835135 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a362e82d-6d26-4dd3-ae58-c172c487b521 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:15.843881 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c077462-3d36-42a1-b0f5-dbe50222d17b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:15.850596 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdcfa6f-be3e-47fa-9518-27c0b52c2a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:15.851324 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdcfa6f-be3e-47fa-9518-27c0b52c2a35 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:15.851608 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdcfa6f-be3e-47fa-9518-27c0b52c2a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:15.851892 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdcfa6f-be3e-47fa-9518-27c0b52c2a35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:15.852665 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdcfa6f-be3e-47fa-9518-27c0b52c2a35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:15.852893 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a362e82d-6d26-4dd3-ae58-c172c487b521 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:15.854360 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdcfa6f-be3e-47fa-9518-27c0b52c2a35 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:15.864732 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a172dd-0ce5-4165-8256-03380254759f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:15.865426 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a172dd-0ce5-4165-8256-03380254759f admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:15.865709 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a172dd-0ce5-4165-8256-03380254759f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:15.865957 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a172dd-0ce5-4165-8256-03380254759f admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:15.866724 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a172dd-0ce5-4165-8256-03380254759f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:15.866917 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1343/2645] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:44:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 21:44:15.868387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a172dd-0ce5-4165-8256-03380254759f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:15.871947 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdcfa6f-be3e-47fa-9518-27c0b52c2a35 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:15.883829 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1303/2646] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:44:15 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:44:15.884049 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdcfa6f-be3e-47fa-9518-27c0b52c2a35 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:15.885019 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a172dd-0ce5-4165-8256-03380254759f admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:15.888268 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da8e9f-95be-4510-ac5b-ee2f8e6a652e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:15.889414 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da8e9f-95be-4510-ac5b-ee2f8e6a652e admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:15.889757 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da8e9f-95be-4510-ac5b-ee2f8e6a652e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:15.890041 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da8e9f-95be-4510-ac5b-ee2f8e6a652e admin admin] PATH_INFO: `/v3/projects/9a4ad8679d5446b591d4083529f992cc/users/b9bd7a1a0e804d23aaa68153549672c9/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:15.896974 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a172dd-0ce5-4165-8256-03380254759f admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:15.903925 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd91d230-45b9-4ab7-8c46-f615d82a3e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:15.903925 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91d230-45b9-4ab7-8c46-f615d82a3e6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:15.903925 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91d230-45b9-4ab7-8c46-f615d82a3e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:15.903925 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91d230-45b9-4ab7-8c46-f615d82a3e6e admin admin] PATH_INFO: `/v3/projects/cf7578de7e1940fe8351ca109f1128a8/users/c27afb93835143fd87b96bb056e634e1/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:15.912004 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da8e9f-95be-4510-ac5b-ee2f8e6a652e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a4ad8679d5446b591d4083529f992cc, user_id=b9bd7a1a0e804d23aaa68153549672c9, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:15.920100 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da8e9f-95be-4510-ac5b-ee2f8e6a652e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:15.922118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd91d230-45b9-4ab7-8c46-f615d82a3e6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf7578de7e1940fe8351ca109f1128a8, user_id=c27afb93835143fd87b96bb056e634e1, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:15.925627 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd91d230-45b9-4ab7-8c46-f615d82a3e6e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:15.940899 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1344/2647] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:44:15 2022] PUT /identity/v3/projects/9a4ad8679d5446b591d4083529f992cc/users/b9bd7a1a0e804d23aaa68153549672c9/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:15.946097 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1304/2648] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:44:15 2022] PUT /identity/v3/projects/cf7578de7e1940fe8351ca109f1128a8/users/c27afb93835143fd87b96bb056e634e1/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:15.946307 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9ae86-e5c6-44f1-bf32-458294ee18d9 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:15.946605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9ae86-e5c6-44f1-bf32-458294ee18d9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:15.947107 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9ae86-e5c6-44f1-bf32-458294ee18d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:15.957374 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7fd783-9355-46bf-abe2-ae2c3468b23d None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:15.957640 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7fd783-9355-46bf-abe2-ae2c3468b23d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:15.957948 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7fd783-9355-46bf-abe2-ae2c3468b23d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:15.965445 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-05a9ae86-e5c6-44f1-bf32-458294ee18d9 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:44:15.967117 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-05a9ae86-e5c6-44f1-bf32-458294ee18d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:15.987871 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-ec7fd783-9355-46bf-abe2-ae2c3468b23d None None] MFA Rules not processed for user `c27afb93835143fd87b96bb056e634e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:44:15.989117 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ec7fd783-9355-46bf-abe2-ae2c3468b23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:16.026542 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:16.033102 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1345/2649] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:44:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:44:16.043346 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a9ae86-e5c6-44f1-bf32-458294ee18d9 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:16.045014 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-05a9ae86-e5c6-44f1-bf32-458294ee18d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:16.064411 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1305/2650] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:44:15 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:44:16.069345 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7fd783-9355-46bf-abe2-ae2c3468b23d None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:16.080432 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7fd783-9355-46bf-abe2-ae2c3468b23d None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:16.085643 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657c155-35b8-4645-9706-5d28d5e32fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:16.086503 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657c155-35b8-4645-9706-5d28d5e32fb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:16.086790 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657c155-35b8-4645-9706-5d28d5e32fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:16.087071 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657c155-35b8-4645-9706-5d28d5e32fb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:16.087778 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d657c155-35b8-4645-9706-5d28d5e32fb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:16.089706 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d657c155-35b8-4645-9706-5d28d5e32fb3 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:16.113852 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a9ae86-e5c6-44f1-bf32-458294ee18d9 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:16.119007 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0944f531-3cb7-4215-97a5-9f392d432826 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:16.120100 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944f531-3cb7-4215-97a5-9f392d432826 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:16.120387 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944f531-3cb7-4215-97a5-9f392d432826 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:16.120662 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944f531-3cb7-4215-97a5-9f392d432826 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:16.125529 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1306/2651] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:44:16 2022] POST /identity/v3/projects => generated 409 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:44:16.128187 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657c155-35b8-4645-9706-5d28d5e32fb3 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:16.135128 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0944f531-3cb7-4215-97a5-9f392d432826 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:16.137910 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0944f531-3cb7-4215-97a5-9f392d432826 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:16.151461 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1346/2652] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:44:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:44:16.200478 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657c155-35b8-4645-9706-5d28d5e32fb3 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:16.205942 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c7bef1-3eb5-4def-9c94-2c454e5430ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:16.206667 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7bef1-3eb5-4def-9c94-2c454e5430ee admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:16.206958 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7bef1-3eb5-4def-9c94-2c454e5430ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:16.207244 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7bef1-3eb5-4def-9c94-2c454e5430ee admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:16.208017 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c7bef1-3eb5-4def-9c94-2c454e5430ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:16.209987 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c7bef1-3eb5-4def-9c94-2c454e5430ee admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:16.233895 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1307/2653] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:44:16 2022] POST /identity/v3/users => generated 311 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:44:16.238420 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0944f531-3cb7-4215-97a5-9f392d432826 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:16.249483 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0944f531-3cb7-4215-97a5-9f392d432826 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:16.254817 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4881d1-c1b3-4ccc-af22-fa62e10a5dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:16.255556 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4881d1-c1b3-4ccc-af22-fa62e10a5dbd admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:16.255852 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4881d1-c1b3-4ccc-af22-fa62e10a5dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:16.256068 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4881d1-c1b3-4ccc-af22-fa62e10a5dbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:16.256984 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4881d1-c1b3-4ccc-af22-fa62e10a5dbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:16.258680 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4881d1-c1b3-4ccc-af22-fa62e10a5dbd admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:16.270138 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1347/2654] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:44:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:44:16.274607 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c7bef1-3eb5-4def-9c94-2c454e5430ee admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:16.285244 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c7bef1-3eb5-4def-9c94-2c454e5430ee admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:16.295388 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f323cc3-b8fe-4ddd-860c-a657889035c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:16.296489 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f323cc3-b8fe-4ddd-860c-a657889035c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:16.296934 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f323cc3-b8fe-4ddd-860c-a657889035c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:16.297314 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f323cc3-b8fe-4ddd-860c-a657889035c3 admin admin] PATH_INFO: `/v3/projects/5c0f7d75d8d34384b2b562b366ec5d93/users/5f1a159026154f25a99eee5a4508223c/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:16.315738 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f323cc3-b8fe-4ddd-860c-a657889035c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c0f7d75d8d34384b2b562b366ec5d93, user_id=5f1a159026154f25a99eee5a4508223c, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:16.319628 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f323cc3-b8fe-4ddd-860c-a657889035c3 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:16.339206 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1308/2655] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:44:16 2022] PUT /identity/v3/projects/5c0f7d75d8d34384b2b562b366ec5d93/users/5f1a159026154f25a99eee5a4508223c/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:16.342867 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4881d1-c1b3-4ccc-af22-fa62e10a5dbd admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:16.354464 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4881d1-c1b3-4ccc-af22-fa62e10a5dbd admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:16.377349 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:16.419707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c154ba2a-860f-4d36-9c62-8718c462ea3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:16.420401 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154ba2a-860f-4d36-9c62-8718c462ea3f admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:16.420714 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154ba2a-860f-4d36-9c62-8718c462ea3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:16.420943 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154ba2a-860f-4d36-9c62-8718c462ea3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:16.421689 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c154ba2a-860f-4d36-9c62-8718c462ea3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:16.423354 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c154ba2a-860f-4d36-9c62-8718c462ea3f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:16.438828 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1348/2656] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:44:16 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:44:16.440571 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f323cc3-b8fe-4ddd-860c-a657889035c3 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:16.454833 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f323cc3-b8fe-4ddd-860c-a657889035c3 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:16.460216 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b591a-b086-422e-b4cc-5232f3ca51e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:16.461190 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b591a-b086-422e-b4cc-5232f3ca51e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:16.461540 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b591a-b086-422e-b4cc-5232f3ca51e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:16.461888 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b591a-b086-422e-b4cc-5232f3ca51e2 admin admin] PATH_INFO: `/v3/projects/5c0f7d75d8d34384b2b562b366ec5d93/users/5f1a159026154f25a99eee5a4508223c/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:16.464148 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b591a-b086-422e-b4cc-5232f3ca51e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c0f7d75d8d34384b2b562b366ec5d93, user_id=5f1a159026154f25a99eee5a4508223c, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:16.467671 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b591a-b086-422e-b4cc-5232f3ca51e2 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:16.490561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1309/2657] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:44:16 2022] PUT /identity/v3/projects/5c0f7d75d8d34384b2b562b366ec5d93/users/5f1a159026154f25a99eee5a4508223c/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:16.501270 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f0b0f-6578-4cbb-bc8f-073a7b4c61bd None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:16.501546 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f0b0f-6578-4cbb-bc8f-073a7b4c61bd None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:16.501850 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f0b0f-6578-4cbb-bc8f-073a7b4c61bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:16.535747 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c83f0b0f-6578-4cbb-bc8f-073a7b4c61bd None None] MFA Rules not processed for user `5f1a159026154f25a99eee5a4508223c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:44:16.536990 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c83f0b0f-6578-4cbb-bc8f-073a7b4c61bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:16.624668 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1349/2658] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:44:16 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:44:16.637195 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b591a-b086-422e-b4cc-5232f3ca51e2 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:16.639805 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-233b591a-b086-422e-b4cc-5232f3ca51e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:16.710913 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b591a-b086-422e-b4cc-5232f3ca51e2 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:16.714953 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd997b6-c47c-4099-a2d7-8ce751752f05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:16.715472 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd997b6-c47c-4099-a2d7-8ce751752f05 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:16.715753 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd997b6-c47c-4099-a2d7-8ce751752f05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:16.716030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd997b6-c47c-4099-a2d7-8ce751752f05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:16.726652 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd997b6-c47c-4099-a2d7-8ce751752f05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:16.728058 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd997b6-c47c-4099-a2d7-8ce751752f05 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:16.739940 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1310/2659] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:44:16 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:44:16.943504 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83f0b0f-6578-4cbb-bc8f-073a7b4c61bd None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:16.954969 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83f0b0f-6578-4cbb-bc8f-073a7b4c61bd None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:16.959741 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8533d508-cdb2-49b5-a25c-77fb81463b6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:16.960439 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8533d508-cdb2-49b5-a25c-77fb81463b6e service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:16.960762 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8533d508-cdb2-49b5-a25c-77fb81463b6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:16.961111 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8533d508-cdb2-49b5-a25c-77fb81463b6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:17.039877 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8533d508-cdb2-49b5-a25c-77fb81463b6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:17.041377 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8533d508-cdb2-49b5-a25c-77fb81463b6e service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:17.052363 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1350/2660] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:44:16 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:44:20.645774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c722e793-59f4-4d28-97a1-d3325ce7f77c None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:20.646384 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c722e793-59f4-4d28-97a1-d3325ce7f77c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:20.646384 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c722e793-59f4-4d28-97a1-d3325ce7f77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:20.659413 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:20.677992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-c722e793-59f4-4d28-97a1-d3325ce7f77c None None] MFA Rules not processed for user `b9bd7a1a0e804d23aaa68153549672c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:44:20.679285 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c722e793-59f4-4d28-97a1-d3325ce7f77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:20.759453 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1311/2661] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:44:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:44:20.774589 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8533d508-cdb2-49b5-a25c-77fb81463b6e service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:20.776379 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8533d508-cdb2-49b5-a25c-77fb81463b6e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:20.828051 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:20.867089 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8533d508-cdb2-49b5-a25c-77fb81463b6e service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:20.872988 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3f6472-a678-462b-82fc-7810ea4737bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:20.873929 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f6472-a678-462b-82fc-7810ea4737bd service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:20.874204 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f6472-a678-462b-82fc-7810ea4737bd service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:20.874512 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f6472-a678-462b-82fc-7810ea4737bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:20.883313 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3f6472-a678-462b-82fc-7810ea4737bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:20.884950 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3f6472-a678-462b-82fc-7810ea4737bd service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:20.895252 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1351/2662] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:44:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:44:22.364311 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4a5d6-c959-4289-baa7-0da82bc049fb None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:22.364868 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4a5d6-c959-4289-baa7-0da82bc049fb None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:22.364970 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4a5d6-c959-4289-baa7-0da82bc049fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:22.384922 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-e7c4a5d6-c959-4289-baa7-0da82bc049fb None None] MFA Rules not processed for user `b9bd7a1a0e804d23aaa68153549672c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:44:22.386178 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e7c4a5d6-c959-4289-baa7-0da82bc049fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:22.393855 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1312/2663] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:44:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:44:23.185341 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3f6472-a678-462b-82fc-7810ea4737bd service nova] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:23.186990 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fb3f6472-a678-462b-82fc-7810ea4737bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:23.278400 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3f6472-a678-462b-82fc-7810ea4737bd service nova] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:23.283628 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c418a62-eea7-4975-a3f0-901f957cd24a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6143bddc05a46ea972709576c7ad95f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:23.284299 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c418a62-eea7-4975-a3f0-901f957cd24a service placement] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:23.284588 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c418a62-eea7-4975-a3f0-901f957cd24a service placement] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:23.284866 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c418a62-eea7-4975-a3f0-901f957cd24a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:23.286724 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7c418a62-eea7-4975-a3f0-901f957cd24a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:23.299225 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c418a62-eea7-4975-a3f0-901f957cd24a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:23.300829 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c418a62-eea7-4975-a3f0-901f957cd24a service placement] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:23.310995 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1352/2664] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:44:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:44:26.781435 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c4a5d6-c959-4289-baa7-0da82bc049fb None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:26.852007 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c4a5d6-c959-4289-baa7-0da82bc049fb None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:26.857383 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c7755-fe89-4cf7-8e56-7d017c2f12fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:26.858519 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c7755-fe89-4cf7-8e56-7d017c2f12fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:26.858855 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c7755-fe89-4cf7-8e56-7d017c2f12fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:26.859067 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c7755-fe89-4cf7-8e56-7d017c2f12fd admin admin] PATH_INFO: `/v3/users/c27afb93835143fd87b96bb056e634e1` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:26.860796 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5c7755-fe89-4cf7-8e56-7d017c2f12fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c27afb93835143fd87b96bb056e634e1)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:26.863422 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5c7755-fe89-4cf7-8e56-7d017c2f12fd admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:26.928462 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1f5c7755-fe89-4cf7-8e56-7d017c2f12fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c27afb93835143fd87b96bb056e634e1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:26.930202 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1f5c7755-fe89-4cf7-8e56-7d017c2f12fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c27afb93835143fd87b96bb056e634e1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:26.952762 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1f5c7755-fe89-4cf7-8e56-7d017c2f12fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c27afb93835143fd87b96bb056e634e1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:26.960801 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1f5c7755-fe89-4cf7-8e56-7d017c2f12fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c27afb93835143fd87b96bb056e634e1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:26.969580 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1f5c7755-fe89-4cf7-8e56-7d017c2f12fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c27afb93835143fd87b96bb056e634e1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:26.976565 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1313/2665] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:44:26 2022] DELETE /identity/v3/users/c27afb93835143fd87b96bb056e634e1 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:26.981306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c26ab-40c7-4d0a-a5be-7370d3c22ecd None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:26.981858 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c26ab-40c7-4d0a-a5be-7370d3c22ecd None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:26.981858 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c26ab-40c7-4d0a-a5be-7370d3c22ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:27.000108 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0a0c26ab-40c7-4d0a-a5be-7370d3c22ecd None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:44:27.001365 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0a0c26ab-40c7-4d0a-a5be-7370d3c22ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:27.069460 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1353/2666] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:44:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:44:27.080818 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c7755-fe89-4cf7-8e56-7d017c2f12fd admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:27.082489 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1f5c7755-fe89-4cf7-8e56-7d017c2f12fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:27.096257 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:27.156312 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c7755-fe89-4cf7-8e56-7d017c2f12fd admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:27.161561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d32010-d23c-47e4-a946-03230c8bf815 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:27.162221 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d32010-d23c-47e4-a946-03230c8bf815 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:27.162531 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d32010-d23c-47e4-a946-03230c8bf815 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:27.162800 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d32010-d23c-47e4-a946-03230c8bf815 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:27.174829 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d32010-d23c-47e4-a946-03230c8bf815 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:27.176281 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d32010-d23c-47e4-a946-03230c8bf815 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:27.187084 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1314/2667] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:44:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:44:27.499149 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c26ab-40c7-4d0a-a5be-7370d3c22ecd None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:27.501016 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0a0c26ab-40c7-4d0a-a5be-7370d3c22ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:27.514948 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c26ab-40c7-4d0a-a5be-7370d3c22ecd None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:27.521725 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa99591-6040-41f1-b1b9-609ce4c45350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:27.522649 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa99591-6040-41f1-b1b9-609ce4c45350 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:27.522945 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa99591-6040-41f1-b1b9-609ce4c45350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:27.523217 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa99591-6040-41f1-b1b9-609ce4c45350 admin admin] PATH_INFO: `/v3/projects/cf7578de7e1940fe8351ca109f1128a8` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:27.524559 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa99591-6040-41f1-b1b9-609ce4c45350 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf7578de7e1940fe8351ca109f1128a8)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:27.526546 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa99591-6040-41f1-b1b9-609ce4c45350 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:27.586023 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5aa99591-6040-41f1-b1b9-609ce4c45350 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf7578de7e1940fe8351ca109f1128a8', 'request_id': 'req-5aa99591-6040-41f1-b1b9-609ce4c45350'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:27.588183 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:27.594260 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5aa99591-6040-41f1-b1b9-609ce4c45350 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf7578de7e1940fe8351ca109f1128a8', 'request_id': 'req-5aa99591-6040-41f1-b1b9-609ce4c45350'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:27.602118 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5aa99591-6040-41f1-b1b9-609ce4c45350 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf7578de7e1940fe8351ca109f1128a8', 'request_id': 'req-5aa99591-6040-41f1-b1b9-609ce4c45350'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:27.623699 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5aa99591-6040-41f1-b1b9-609ce4c45350 admin admin] The token cache is being invalidate because project cf7578de7e1940fe8351ca109f1128a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:44:27.624003 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5aa99591-6040-41f1-b1b9-609ce4c45350 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:27.624953 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5aa99591-6040-41f1-b1b9-609ce4c45350 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:27.627202 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1354/2668] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:44:27 2022] DELETE /identity/v3/projects/cf7578de7e1940fe8351ca109f1128a8 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:27.632042 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d32010-d23c-47e4-a946-03230c8bf815 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:27.645257 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d32010-d23c-47e4-a946-03230c8bf815 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:27.713934 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b11651-f4ea-4d6a-8d53-5cace0b939a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:27.714780 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b11651-f4ea-4d6a-8d53-5cace0b939a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:27.715160 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b11651-f4ea-4d6a-8d53-5cace0b939a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:27.716310 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b11651-f4ea-4d6a-8d53-5cace0b939a1 admin admin] PATH_INFO: `/v3/users/5f1a159026154f25a99eee5a4508223c` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:27.716921 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b11651-f4ea-4d6a-8d53-5cace0b939a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f1a159026154f25a99eee5a4508223c)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:27.718634 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b11651-f4ea-4d6a-8d53-5cace0b939a1 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:27.776046 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-29b11651-f4ea-4d6a-8d53-5cace0b939a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f1a159026154f25a99eee5a4508223c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:27.777061 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-29b11651-f4ea-4d6a-8d53-5cace0b939a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f1a159026154f25a99eee5a4508223c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:27.792945 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-29b11651-f4ea-4d6a-8d53-5cace0b939a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f1a159026154f25a99eee5a4508223c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:27.801369 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-29b11651-f4ea-4d6a-8d53-5cace0b939a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f1a159026154f25a99eee5a4508223c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:27.811559 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-29b11651-f4ea-4d6a-8d53-5cace0b939a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f1a159026154f25a99eee5a4508223c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:27.820065 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1315/2669] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:44:27 2022] DELETE /identity/v3/users/5f1a159026154f25a99eee5a4508223c => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:28.129130 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa99591-6040-41f1-b1b9-609ce4c45350 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:28.130770 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5aa99591-6040-41f1-b1b9-609ce4c45350 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:28.148288 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa99591-6040-41f1-b1b9-609ce4c45350 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:28.219208 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a1121-2fc8-4fcb-949f-1272762e16a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:28.220101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a1121-2fc8-4fcb-949f-1272762e16a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:28.220381 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a1121-2fc8-4fcb-949f-1272762e16a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:28.220655 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a1121-2fc8-4fcb-949f-1272762e16a7 admin admin] PATH_INFO: `/v3/projects/5c0f7d75d8d34384b2b562b366ec5d93` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:28.221868 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37a1121-2fc8-4fcb-949f-1272762e16a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c0f7d75d8d34384b2b562b366ec5d93)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:28.223905 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37a1121-2fc8-4fcb-949f-1272762e16a7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:28.271352 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b37a1121-2fc8-4fcb-949f-1272762e16a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c0f7d75d8d34384b2b562b366ec5d93', 'request_id': 'req-b37a1121-2fc8-4fcb-949f-1272762e16a7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:28.277825 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b37a1121-2fc8-4fcb-949f-1272762e16a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c0f7d75d8d34384b2b562b366ec5d93', 'request_id': 'req-b37a1121-2fc8-4fcb-949f-1272762e16a7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:28.284295 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b37a1121-2fc8-4fcb-949f-1272762e16a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c0f7d75d8d34384b2b562b366ec5d93', 'request_id': 'req-b37a1121-2fc8-4fcb-949f-1272762e16a7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:28.309800 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b37a1121-2fc8-4fcb-949f-1272762e16a7 admin admin] The token cache is being invalidate because project 5c0f7d75d8d34384b2b562b366ec5d93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:44:28.310082 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b37a1121-2fc8-4fcb-949f-1272762e16a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:28.310961 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b37a1121-2fc8-4fcb-949f-1272762e16a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:28.313237 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1355/2670] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:44:28 2022] DELETE /identity/v3/projects/5c0f7d75d8d34384b2b562b366ec5d93 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:28.319993 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c048ae2-b7e8-497d-ad2e-ed3cae907323 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:28.320317 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c048ae2-b7e8-497d-ad2e-ed3cae907323 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:28.320593 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c048ae2-b7e8-497d-ad2e-ed3cae907323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:28.339429 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-0c048ae2-b7e8-497d-ad2e-ed3cae907323 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:44:28.340642 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0c048ae2-b7e8-497d-ad2e-ed3cae907323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:28.368597 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:28.409283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1316/2671] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:44:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:44:28.417369 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84533d-52f7-4581-b78a-ac9971363a83 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:28.417646 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84533d-52f7-4581-b78a-ac9971363a83 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:28.417903 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84533d-52f7-4581-b78a-ac9971363a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:28.437595 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0e84533d-52f7-4581-b78a-ac9971363a83 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:44:28.438838 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0e84533d-52f7-4581-b78a-ac9971363a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:28.448501 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1356/2672] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:44:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:44:28.453189 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c048ae2-b7e8-497d-ad2e-ed3cae907323 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:28.464086 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c048ae2-b7e8-497d-ad2e-ed3cae907323 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:28.471217 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b055e-e6c4-4aa5-8076-36b3bc4a37ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:28.471985 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b055e-e6c4-4aa5-8076-36b3bc4a37ca admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:28.472343 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b055e-e6c4-4aa5-8076-36b3bc4a37ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:28.472765 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b055e-e6c4-4aa5-8076-36b3bc4a37ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:28.473932 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b055e-e6c4-4aa5-8076-36b3bc4a37ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:44:28.474452 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b055e-e6c4-4aa5-8076-36b3bc4a37ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:28.476984 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b055e-e6c4-4aa5-8076-36b3bc4a37ca admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:28.491498 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1317/2673] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:44:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:44:28.496168 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e84533d-52f7-4581-b78a-ac9971363a83 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:28.506070 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e84533d-52f7-4581-b78a-ac9971363a83 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:28.512124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2754c90d-586b-4273-bc6b-310ad0af97f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:28.512790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2754c90d-586b-4273-bc6b-310ad0af97f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:28.513072 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2754c90d-586b-4273-bc6b-310ad0af97f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:28.513447 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2754c90d-586b-4273-bc6b-310ad0af97f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:28.514175 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2754c90d-586b-4273-bc6b-310ad0af97f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:28.516190 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2754c90d-586b-4273-bc6b-310ad0af97f0 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:28.547952 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1357/2674] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:44:28 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:44:28.552092 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b055e-e6c4-4aa5-8076-36b3bc4a37ca admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:28.562911 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b055e-e6c4-4aa5-8076-36b3bc4a37ca admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:28.627675 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5656fc2e-09a3-44a3-98ba-c344e53c7155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:28.628352 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5656fc2e-09a3-44a3-98ba-c344e53c7155 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:28.628632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5656fc2e-09a3-44a3-98ba-c344e53c7155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:28.628905 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5656fc2e-09a3-44a3-98ba-c344e53c7155 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:28.629610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5656fc2e-09a3-44a3-98ba-c344e53c7155 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:28.631611 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5656fc2e-09a3-44a3-98ba-c344e53c7155 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:28.654871 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1318/2675] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:44:28 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:44:28.660983 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2754c90d-586b-4273-bc6b-310ad0af97f0 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:28.676488 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2754c90d-586b-4273-bc6b-310ad0af97f0 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:28.684041 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ca2c10-ab2f-41e0-8ebb-0a266f19d278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:28.685042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca2c10-ab2f-41e0-8ebb-0a266f19d278 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:28.685749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca2c10-ab2f-41e0-8ebb-0a266f19d278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:28.685988 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca2c10-ab2f-41e0-8ebb-0a266f19d278 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:28.686785 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ca2c10-ab2f-41e0-8ebb-0a266f19d278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:28.688378 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ca2c10-ab2f-41e0-8ebb-0a266f19d278 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:28.699934 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1358/2676] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:44:28.704331 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5656fc2e-09a3-44a3-98ba-c344e53c7155 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:28.715237 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5656fc2e-09a3-44a3-98ba-c344e53c7155 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:28.721113 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5734e0d-13b5-4d36-b289-c206653a0ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:28.721818 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5734e0d-13b5-4d36-b289-c206653a0ade admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:28.722225 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5734e0d-13b5-4d36-b289-c206653a0ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:28.722533 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5734e0d-13b5-4d36-b289-c206653a0ade admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:28.723282 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5734e0d-13b5-4d36-b289-c206653a0ade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:28.724927 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5734e0d-13b5-4d36-b289-c206653a0ade admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:28.736179 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1319/2677] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:44:28.740731 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ca2c10-ab2f-41e0-8ebb-0a266f19d278 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:28.751227 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ca2c10-ab2f-41e0-8ebb-0a266f19d278 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:28.756335 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555254eb-85ed-48da-9c3e-0bef23ff0c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:28.757231 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555254eb-85ed-48da-9c3e-0bef23ff0c19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:28.757500 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555254eb-85ed-48da-9c3e-0bef23ff0c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:28.757770 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555254eb-85ed-48da-9c3e-0bef23ff0c19 admin admin] PATH_INFO: `/v3/projects/6c8353db1a85474bbd619cba9816228f/users/b97643b0742d4ed498af35c157c9b3da/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:28.774411 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555254eb-85ed-48da-9c3e-0bef23ff0c19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c8353db1a85474bbd619cba9816228f, user_id=b97643b0742d4ed498af35c157c9b3da, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:28.777475 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555254eb-85ed-48da-9c3e-0bef23ff0c19 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:28.791497 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1359/2678] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:44:28 2022] PUT /identity/v3/projects/6c8353db1a85474bbd619cba9816228f/users/b97643b0742d4ed498af35c157c9b3da/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:28.802155 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfabd093-6073-4a5d-a479-2f1952794ae3 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:28.802666 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfabd093-6073-4a5d-a479-2f1952794ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:28.802666 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfabd093-6073-4a5d-a479-2f1952794ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:28.833474 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-cfabd093-6073-4a5d-a479-2f1952794ae3 None None] MFA Rules not processed for user `b97643b0742d4ed498af35c157c9b3da`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:44:28.834728 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-cfabd093-6073-4a5d-a479-2f1952794ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:28.907416 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1320/2679] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:44:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:44:28.912265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555254eb-85ed-48da-9c3e-0bef23ff0c19 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:28.927681 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:28.986549 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555254eb-85ed-48da-9c3e-0bef23ff0c19 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:28.991479 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7591c82e-b402-40e1-bdda-546dbb0805c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:28.992159 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591c82e-b402-40e1-bdda-546dbb0805c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:28.992450 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591c82e-b402-40e1-bdda-546dbb0805c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:28.992727 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591c82e-b402-40e1-bdda-546dbb0805c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:28.993485 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7591c82e-b402-40e1-bdda-546dbb0805c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:28.995451 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7591c82e-b402-40e1-bdda-546dbb0805c9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:29.038142 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1360/2680] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:44:28 2022] POST /identity/v3/projects => generated 387 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:44:29.041015 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfabd093-6073-4a5d-a479-2f1952794ae3 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:29.054635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfabd093-6073-4a5d-a479-2f1952794ae3 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:29.090653 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:29.133829 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d09f75-1dfb-485b-912b-451d8c16d749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:29.134557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d09f75-1dfb-485b-912b-451d8c16d749 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:29.134845 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d09f75-1dfb-485b-912b-451d8c16d749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:29.135126 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d09f75-1dfb-485b-912b-451d8c16d749 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:29.135826 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d09f75-1dfb-485b-912b-451d8c16d749 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:29.137925 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d09f75-1dfb-485b-912b-451d8c16d749 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:29.160118 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1321/2681] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:44:29 2022] POST /identity/v3/users => generated 300 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:44:29.164255 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7591c82e-b402-40e1-bdda-546dbb0805c9 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:29.175759 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7591c82e-b402-40e1-bdda-546dbb0805c9 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:29.180796 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f22bd-abf1-4c76-956f-0b7e87385d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:29.181477 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f22bd-abf1-4c76-956f-0b7e87385d47 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:29.181726 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f22bd-abf1-4c76-956f-0b7e87385d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:29.182000 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f22bd-abf1-4c76-956f-0b7e87385d47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:29.182775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597f22bd-abf1-4c76-956f-0b7e87385d47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:29.184382 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597f22bd-abf1-4c76-956f-0b7e87385d47 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:29.203592 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1361/2682] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:44:29.205463 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d09f75-1dfb-485b-912b-451d8c16d749 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:29.220296 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d09f75-1dfb-485b-912b-451d8c16d749 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:29.225735 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160f171-a1fa-4bea-a0cf-4118e38a151b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:29.226682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160f171-a1fa-4bea-a0cf-4118e38a151b admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:29.226968 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160f171-a1fa-4bea-a0cf-4118e38a151b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:29.227246 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160f171-a1fa-4bea-a0cf-4118e38a151b admin admin] PATH_INFO: `/v3/projects/ec457805811d4ffbade1306dd7e79198/users/7a7bc51e15fc4ac3a1a6c8094bda6d73/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:29.244433 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c160f171-a1fa-4bea-a0cf-4118e38a151b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec457805811d4ffbade1306dd7e79198, user_id=7a7bc51e15fc4ac3a1a6c8094bda6d73, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:29.247325 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c160f171-a1fa-4bea-a0cf-4118e38a151b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:29.262843 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1322/2683] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:44:29 2022] PUT /identity/v3/projects/ec457805811d4ffbade1306dd7e79198/users/7a7bc51e15fc4ac3a1a6c8094bda6d73/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:29.267070 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f22bd-abf1-4c76-956f-0b7e87385d47 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:29.277875 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f22bd-abf1-4c76-956f-0b7e87385d47 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:29.339401 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed9cbc-7545-495f-b655-c9fafe4194c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:29.339960 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed9cbc-7545-495f-b655-c9fafe4194c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:29.340241 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed9cbc-7545-495f-b655-c9fafe4194c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:29.340552 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed9cbc-7545-495f-b655-c9fafe4194c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:29.341289 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ed9cbc-7545-495f-b655-c9fafe4194c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:29.343170 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ed9cbc-7545-495f-b655-c9fafe4194c4 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:29.357190 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1362/2684] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:44:29.358462 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160f171-a1fa-4bea-a0cf-4118e38a151b admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:29.369088 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160f171-a1fa-4bea-a0cf-4118e38a151b admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:29.374232 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d82cc-8194-4db4-bc65-dc6352a6db81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:29.375163 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d82cc-8194-4db4-bc65-dc6352a6db81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:29.375438 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d82cc-8194-4db4-bc65-dc6352a6db81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:29.375742 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d82cc-8194-4db4-bc65-dc6352a6db81 admin admin] PATH_INFO: `/v3/projects/ec457805811d4ffbade1306dd7e79198/users/7a7bc51e15fc4ac3a1a6c8094bda6d73/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:29.377796 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359d82cc-8194-4db4-bc65-dc6352a6db81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec457805811d4ffbade1306dd7e79198, user_id=7a7bc51e15fc4ac3a1a6c8094bda6d73, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:29.380496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359d82cc-8194-4db4-bc65-dc6352a6db81 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:29.391642 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1323/2685] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:44:29 2022] PUT /identity/v3/projects/ec457805811d4ffbade1306dd7e79198/users/7a7bc51e15fc4ac3a1a6c8094bda6d73/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:29.403164 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9312e0-50cb-4b19-8766-9187b2faa4f0 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:29.403451 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9312e0-50cb-4b19-8766-9187b2faa4f0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:29.403732 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9312e0-50cb-4b19-8766-9187b2faa4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:29.432126 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-df9312e0-50cb-4b19-8766-9187b2faa4f0 None None] MFA Rules not processed for user `7a7bc51e15fc4ac3a1a6c8094bda6d73`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:44:29.433777 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-df9312e0-50cb-4b19-8766-9187b2faa4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:29.507896 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1363/2686] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:44:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:44:29.519063 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d82cc-8194-4db4-bc65-dc6352a6db81 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:29.521014 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-359d82cc-8194-4db4-bc65-dc6352a6db81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:29.600903 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d82cc-8194-4db4-bc65-dc6352a6db81 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:29.604757 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73962d3-9d78-424c-9657-489ef816a37f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:29.605452 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73962d3-9d78-424c-9657-489ef816a37f service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:29.605735 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73962d3-9d78-424c-9657-489ef816a37f service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:29.606014 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73962d3-9d78-424c-9657-489ef816a37f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:29.616498 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73962d3-9d78-424c-9657-489ef816a37f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:29.617944 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73962d3-9d78-424c-9657-489ef816a37f service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:29.628215 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1324/2687] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:44:29 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:44:30.239497 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9312e0-50cb-4b19-8766-9187b2faa4f0 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:30.244251 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:30.308880 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9312e0-50cb-4b19-8766-9187b2faa4f0 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:30.314597 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879af8d-ea7a-4194-a279-6deede117e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:30.315526 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879af8d-ea7a-4194-a279-6deede117e69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:30.315887 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879af8d-ea7a-4194-a279-6deede117e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:30.316091 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879af8d-ea7a-4194-a279-6deede117e69 admin admin] PATH_INFO: `/v3/users/b97643b0742d4ed498af35c157c9b3da` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:30.317377 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9879af8d-ea7a-4194-a279-6deede117e69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b97643b0742d4ed498af35c157c9b3da)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:30.319724 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9879af8d-ea7a-4194-a279-6deede117e69 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:30.368800 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9879af8d-ea7a-4194-a279-6deede117e69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b97643b0742d4ed498af35c157c9b3da'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:30.383218 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9879af8d-ea7a-4194-a279-6deede117e69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b97643b0742d4ed498af35c157c9b3da'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:30.383851 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9879af8d-ea7a-4194-a279-6deede117e69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b97643b0742d4ed498af35c157c9b3da'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:30.384470 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9879af8d-ea7a-4194-a279-6deede117e69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b97643b0742d4ed498af35c157c9b3da'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:30.392788 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9879af8d-ea7a-4194-a279-6deede117e69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b97643b0742d4ed498af35c157c9b3da'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:30.408575 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1364/2688] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:44:30 2022] DELETE /identity/v3/users/b97643b0742d4ed498af35c157c9b3da => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:30.414360 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28ca77-490c-49eb-a48a-daf8181715c6 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:30.415185 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28ca77-490c-49eb-a48a-daf8181715c6 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:30.415185 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28ca77-490c-49eb-a48a-daf8181715c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:30.435171 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-1f28ca77-490c-49eb-a48a-daf8181715c6 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:44:30.436386 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1f28ca77-490c-49eb-a48a-daf8181715c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:30.505051 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1325/2689] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:44:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:44:30.517018 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879af8d-ea7a-4194-a279-6deede117e69 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:30.518561 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9879af8d-ea7a-4194-a279-6deede117e69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:30.586264 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879af8d-ea7a-4194-a279-6deede117e69 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:30.590178 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8941974-fa4f-4e7f-9c21-8f14dc0da099 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:30.591065 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8941974-fa4f-4e7f-9c21-8f14dc0da099 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:30.591418 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8941974-fa4f-4e7f-9c21-8f14dc0da099 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:30.591705 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8941974-fa4f-4e7f-9c21-8f14dc0da099 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:30.602320 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8941974-fa4f-4e7f-9c21-8f14dc0da099 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:30.603793 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8941974-fa4f-4e7f-9c21-8f14dc0da099 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:30.613910 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1365/2690] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:44:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:44:30.909370 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f28ca77-490c-49eb-a48a-daf8181715c6 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:30.910941 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1f28ca77-490c-49eb-a48a-daf8181715c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:30.918955 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:30.927504 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f28ca77-490c-49eb-a48a-daf8181715c6 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:30.934040 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f97d27-1b87-473b-aef0-c4b5862663c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:30.935615 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f97d27-1b87-473b-aef0-c4b5862663c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:30.935963 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f97d27-1b87-473b-aef0-c4b5862663c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:30.936222 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f97d27-1b87-473b-aef0-c4b5862663c0 admin admin] PATH_INFO: `/v3/projects/6c8353db1a85474bbd619cba9816228f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:30.937647 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f97d27-1b87-473b-aef0-c4b5862663c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c8353db1a85474bbd619cba9816228f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:30.939429 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f97d27-1b87-473b-aef0-c4b5862663c0 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:30.988906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-32f97d27-1b87-473b-aef0-c4b5862663c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c8353db1a85474bbd619cba9816228f', 'request_id': 'req-32f97d27-1b87-473b-aef0-c4b5862663c0'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:30.995117 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-32f97d27-1b87-473b-aef0-c4b5862663c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c8353db1a85474bbd619cba9816228f', 'request_id': 'req-32f97d27-1b87-473b-aef0-c4b5862663c0'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:31.004463 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-32f97d27-1b87-473b-aef0-c4b5862663c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c8353db1a85474bbd619cba9816228f', 'request_id': 'req-32f97d27-1b87-473b-aef0-c4b5862663c0'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:31.015338 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-32f97d27-1b87-473b-aef0-c4b5862663c0 admin admin] The token cache is being invalidate because project 6c8353db1a85474bbd619cba9816228f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:44:31.015611 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-32f97d27-1b87-473b-aef0-c4b5862663c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:31.016243 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-32f97d27-1b87-473b-aef0-c4b5862663c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:31.017841 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1326/2691] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:44:30 2022] DELETE /identity/v3/projects/6c8353db1a85474bbd619cba9816228f => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:31.022465 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8941974-fa4f-4e7f-9c21-8f14dc0da099 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:31.032654 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8941974-fa4f-4e7f-9c21-8f14dc0da099 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:31.053605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:31.099761 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d13c5-617f-49c1-9925-64506cfb8a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:31.100696 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d13c5-617f-49c1-9925-64506cfb8a8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:31.100981 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d13c5-617f-49c1-9925-64506cfb8a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:31.101256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d13c5-617f-49c1-9925-64506cfb8a8c admin admin] PATH_INFO: `/v3/users/7a7bc51e15fc4ac3a1a6c8094bda6d73` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:31.103402 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d13c5-617f-49c1-9925-64506cfb8a8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a7bc51e15fc4ac3a1a6c8094bda6d73)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:31.106043 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d13c5-617f-49c1-9925-64506cfb8a8c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:31.157553 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7b3d13c5-617f-49c1-9925-64506cfb8a8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a7bc51e15fc4ac3a1a6c8094bda6d73'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:31.171430 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7b3d13c5-617f-49c1-9925-64506cfb8a8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a7bc51e15fc4ac3a1a6c8094bda6d73'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:31.172205 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7b3d13c5-617f-49c1-9925-64506cfb8a8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a7bc51e15fc4ac3a1a6c8094bda6d73'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:31.172863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7b3d13c5-617f-49c1-9925-64506cfb8a8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a7bc51e15fc4ac3a1a6c8094bda6d73'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:31.180582 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7b3d13c5-617f-49c1-9925-64506cfb8a8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a7bc51e15fc4ac3a1a6c8094bda6d73'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:31.196644 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1366/2692] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:44:31 2022] DELETE /identity/v3/users/7a7bc51e15fc4ac3a1a6c8094bda6d73 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:31.496367 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f97d27-1b87-473b-aef0-c4b5862663c0 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:31.498687 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-32f97d27-1b87-473b-aef0-c4b5862663c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:31.513283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f97d27-1b87-473b-aef0-c4b5862663c0 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:31.572901 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741f4997-49fa-4144-a814-a473d964f081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:31.574151 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f4997-49fa-4144-a814-a473d964f081 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:31.574637 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f4997-49fa-4144-a814-a473d964f081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:31.574908 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f4997-49fa-4144-a814-a473d964f081 admin admin] PATH_INFO: `/v3/projects/ec457805811d4ffbade1306dd7e79198` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:31.576617 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741f4997-49fa-4144-a814-a473d964f081 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec457805811d4ffbade1306dd7e79198)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:31.578519 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741f4997-49fa-4144-a814-a473d964f081 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:31.626014 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-741f4997-49fa-4144-a814-a473d964f081 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec457805811d4ffbade1306dd7e79198', 'request_id': 'req-741f4997-49fa-4144-a814-a473d964f081'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:31.632239 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-741f4997-49fa-4144-a814-a473d964f081 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec457805811d4ffbade1306dd7e79198', 'request_id': 'req-741f4997-49fa-4144-a814-a473d964f081'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:31.642878 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-741f4997-49fa-4144-a814-a473d964f081 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec457805811d4ffbade1306dd7e79198', 'request_id': 'req-741f4997-49fa-4144-a814-a473d964f081'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:31.655041 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-741f4997-49fa-4144-a814-a473d964f081 admin admin] The token cache is being invalidate because project ec457805811d4ffbade1306dd7e79198 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:44:31.655328 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-741f4997-49fa-4144-a814-a473d964f081 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:31.656058 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-741f4997-49fa-4144-a814-a473d964f081 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:31.657672 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1327/2693] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:44:31 2022] DELETE /identity/v3/projects/ec457805811d4ffbade1306dd7e79198 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:31.662846 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8e912-857e-419a-9e8c-b801ffd414b1 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:31.663366 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8e912-857e-419a-9e8c-b801ffd414b1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:31.663567 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8e912-857e-419a-9e8c-b801ffd414b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:31.682340 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-6df8e912-857e-419a-9e8c-b801ffd414b1 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:44:31.683568 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6df8e912-857e-419a-9e8c-b801ffd414b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:31.747467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1367/2694] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:44:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:44:31.756634 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c60e7-9fe2-4ac6-a44a-84c56bc6718b None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:31.757008 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c60e7-9fe2-4ac6-a44a-84c56bc6718b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:31.757282 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c60e7-9fe2-4ac6-a44a-84c56bc6718b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:31.779340 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-656c60e7-9fe2-4ac6-a44a-84c56bc6718b None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:44:31.780657 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-656c60e7-9fe2-4ac6-a44a-84c56bc6718b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:31.789631 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1328/2695] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:44:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:44:31.794488 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df8e912-857e-419a-9e8c-b801ffd414b1 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:31.804821 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df8e912-857e-419a-9e8c-b801ffd414b1 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:31.809708 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ffcd7d-cf0d-41a7-9a7d-fbaded5ec695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:31.810622 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ffcd7d-cf0d-41a7-9a7d-fbaded5ec695 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:31.810765 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ffcd7d-cf0d-41a7-9a7d-fbaded5ec695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:31.811063 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ffcd7d-cf0d-41a7-9a7d-fbaded5ec695 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:31.811799 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ffcd7d-cf0d-41a7-9a7d-fbaded5ec695 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:44:31.812109 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ffcd7d-cf0d-41a7-9a7d-fbaded5ec695 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:31.813679 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ffcd7d-cf0d-41a7-9a7d-fbaded5ec695 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:31.827241 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1368/2696] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:44:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:44:31.831914 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656c60e7-9fe2-4ac6-a44a-84c56bc6718b None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:31.842127 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656c60e7-9fe2-4ac6-a44a-84c56bc6718b None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:31.848143 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b044715-91b5-4d01-9bac-6338ed9a04a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:31.848900 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b044715-91b5-4d01-9bac-6338ed9a04a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:31.849206 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b044715-91b5-4d01-9bac-6338ed9a04a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:31.849465 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b044715-91b5-4d01-9bac-6338ed9a04a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:31.850165 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b044715-91b5-4d01-9bac-6338ed9a04a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:31.852152 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b044715-91b5-4d01-9bac-6338ed9a04a4 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:31.857051 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:31.881970 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1329/2697] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:44:31 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:44:31.886463 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ffcd7d-cf0d-41a7-9a7d-fbaded5ec695 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:31.896876 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ffcd7d-cf0d-41a7-9a7d-fbaded5ec695 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:31.928822 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:31.961948 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d4932-9670-493f-bbbe-c52a2e59f24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:31.962695 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d4932-9670-493f-bbbe-c52a2e59f24c admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:31.963004 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d4932-9670-493f-bbbe-c52a2e59f24c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:31.963282 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d4932-9670-493f-bbbe-c52a2e59f24c admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:31.963975 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267d4932-9670-493f-bbbe-c52a2e59f24c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:31.965902 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267d4932-9670-493f-bbbe-c52a2e59f24c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:31.993617 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1369/2698] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:44:31 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:44:31.995945 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b044715-91b5-4d01-9bac-6338ed9a04a4 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:32.011042 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b044715-91b5-4d01-9bac-6338ed9a04a4 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:32.016091 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73dc5d3-ad55-4541-ab35-3ffefec50f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:32.016846 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73dc5d3-ad55-4541-ab35-3ffefec50f8c admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:32.017354 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73dc5d3-ad55-4541-ab35-3ffefec50f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:32.017708 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73dc5d3-ad55-4541-ab35-3ffefec50f8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:32.018558 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73dc5d3-ad55-4541-ab35-3ffefec50f8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:32.020235 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73dc5d3-ad55-4541-ab35-3ffefec50f8c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:32.033188 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1330/2699] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:44:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 21:44:32.037848 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d4932-9670-493f-bbbe-c52a2e59f24c admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:32.048588 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d4932-9670-493f-bbbe-c52a2e59f24c admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:32.054543 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544a19e-28d0-4535-9d8f-edd8ca11496a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:32.055214 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544a19e-28d0-4535-9d8f-edd8ca11496a admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:32.055494 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544a19e-28d0-4535-9d8f-edd8ca11496a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:32.055769 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544a19e-28d0-4535-9d8f-edd8ca11496a admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:32.056517 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f544a19e-28d0-4535-9d8f-edd8ca11496a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:32.058158 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f544a19e-28d0-4535-9d8f-edd8ca11496a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:32.070850 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1370/2700] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:44:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:44:32.075004 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73dc5d3-ad55-4541-ab35-3ffefec50f8c admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:32.085887 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73dc5d3-ad55-4541-ab35-3ffefec50f8c admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:32.091939 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965db147-557c-4f35-95f9-ade759a13d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:32.092835 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965db147-557c-4f35-95f9-ade759a13d75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:32.093116 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965db147-557c-4f35-95f9-ade759a13d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:32.093397 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965db147-557c-4f35-95f9-ade759a13d75 admin admin] PATH_INFO: `/v3/projects/77190064c2bc49808a1a694da39903f1/users/0fdd4c26592c417ca5bd739739708b06/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:32.109710 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965db147-557c-4f35-95f9-ade759a13d75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77190064c2bc49808a1a694da39903f1, user_id=0fdd4c26592c417ca5bd739739708b06, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:32.112414 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965db147-557c-4f35-95f9-ade759a13d75 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:32.127352 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1331/2701] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:44:32 2022] PUT /identity/v3/projects/77190064c2bc49808a1a694da39903f1/users/0fdd4c26592c417ca5bd739739708b06/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:32.137971 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd5d02-86e6-423b-9b63-e3e9701cb5c0 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:32.138267 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd5d02-86e6-423b-9b63-e3e9701cb5c0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:32.139197 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd5d02-86e6-423b-9b63-e3e9701cb5c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:32.170933 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a6bd5d02-86e6-423b-9b63-e3e9701cb5c0 None None] MFA Rules not processed for user `0fdd4c26592c417ca5bd739739708b06`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:44:32.172217 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a6bd5d02-86e6-423b-9b63-e3e9701cb5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:32.243404 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1371/2702] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:44:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:44:32.248373 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965db147-557c-4f35-95f9-ade759a13d75 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:32.322338 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965db147-557c-4f35-95f9-ade759a13d75 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:32.328522 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6d419a-121b-46a8-8c9c-4b839796b61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:32.329233 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d419a-121b-46a8-8c9c-4b839796b61c admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:32.329713 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d419a-121b-46a8-8c9c-4b839796b61c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:32.330314 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d419a-121b-46a8-8c9c-4b839796b61c admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:32.331558 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6d419a-121b-46a8-8c9c-4b839796b61c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:32.334617 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6d419a-121b-46a8-8c9c-4b839796b61c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:32.368347 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1332/2703] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:44:32 2022] POST /identity/v3/projects => generated 379 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:44:32.372696 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bd5d02-86e6-423b-9b63-e3e9701cb5c0 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:32.383528 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bd5d02-86e6-423b-9b63-e3e9701cb5c0 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:32.441059 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcf1ee-56f6-4d87-991d-9757cb0fae6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:32.441741 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcf1ee-56f6-4d87-991d-9757cb0fae6a admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:32.442021 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcf1ee-56f6-4d87-991d-9757cb0fae6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:32.442391 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcf1ee-56f6-4d87-991d-9757cb0fae6a admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:32.443156 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbcf1ee-56f6-4d87-991d-9757cb0fae6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:32.445334 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbcf1ee-56f6-4d87-991d-9757cb0fae6a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:32.467559 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1372/2704] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:44:32 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:44:32.471870 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6d419a-121b-46a8-8c9c-4b839796b61c admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:32.482896 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6d419a-121b-46a8-8c9c-4b839796b61c admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:32.487892 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9419c6fc-fdcd-4294-beb2-ec8101223fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:32.488606 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419c6fc-fdcd-4294-beb2-ec8101223fd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:32.488887 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419c6fc-fdcd-4294-beb2-ec8101223fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:32.489160 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419c6fc-fdcd-4294-beb2-ec8101223fd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:32.489887 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9419c6fc-fdcd-4294-beb2-ec8101223fd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:32.491528 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9419c6fc-fdcd-4294-beb2-ec8101223fd5 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:32.503915 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1333/2705] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:44:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:44:32.508386 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcf1ee-56f6-4d87-991d-9757cb0fae6a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:32.519233 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcf1ee-56f6-4d87-991d-9757cb0fae6a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:32.524049 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf97598-b07b-48d7-8710-32eab1fe1ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:32.524951 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf97598-b07b-48d7-8710-32eab1fe1ba1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:32.525230 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf97598-b07b-48d7-8710-32eab1fe1ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:32.525505 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf97598-b07b-48d7-8710-32eab1fe1ba1 admin admin] PATH_INFO: `/v3/projects/a5a8a5cab0d24e7884d751c46a5b62a5/users/497711da8c3146d6888a19e07e9b1cae/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:32.541895 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf97598-b07b-48d7-8710-32eab1fe1ba1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5a8a5cab0d24e7884d751c46a5b62a5, user_id=497711da8c3146d6888a19e07e9b1cae, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:32.544817 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf97598-b07b-48d7-8710-32eab1fe1ba1 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:32.559739 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1373/2706] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:44:32 2022] PUT /identity/v3/projects/a5a8a5cab0d24e7884d751c46a5b62a5/users/497711da8c3146d6888a19e07e9b1cae/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:32.564050 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9419c6fc-fdcd-4294-beb2-ec8101223fd5 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:32.577381 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9419c6fc-fdcd-4294-beb2-ec8101223fd5 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:32.652594 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666ebdc0-e262-4671-b284-ebd9e0f00ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:32.653775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ebdc0-e262-4671-b284-ebd9e0f00ece admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:32.653775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ebdc0-e262-4671-b284-ebd9e0f00ece admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:32.654211 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ebdc0-e262-4671-b284-ebd9e0f00ece admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:32.655197 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666ebdc0-e262-4671-b284-ebd9e0f00ece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:32.657297 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666ebdc0-e262-4671-b284-ebd9e0f00ece admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:32.670292 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1334/2707] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:44:32 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:44:32.675691 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf97598-b07b-48d7-8710-32eab1fe1ba1 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:32.682920 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:32.687513 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf97598-b07b-48d7-8710-32eab1fe1ba1 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:32.693829 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59cac9-d236-4866-ab19-692c14f32efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:32.694881 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59cac9-d236-4866-ab19-692c14f32efe admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:32.695233 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59cac9-d236-4866-ab19-692c14f32efe admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:32.695579 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59cac9-d236-4866-ab19-692c14f32efe admin admin] PATH_INFO: `/v3/projects/a5a8a5cab0d24e7884d751c46a5b62a5/users/497711da8c3146d6888a19e07e9b1cae/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:32.697673 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d59cac9-d236-4866-ab19-692c14f32efe admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5a8a5cab0d24e7884d751c46a5b62a5, user_id=497711da8c3146d6888a19e07e9b1cae, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:32.700461 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d59cac9-d236-4866-ab19-692c14f32efe admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:32.712729 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1374/2708] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:44:32 2022] PUT /identity/v3/projects/a5a8a5cab0d24e7884d751c46a5b62a5/users/497711da8c3146d6888a19e07e9b1cae/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:32.724010 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0252c71-193e-4bfa-a394-3341c67d5bd7 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:32.724308 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0252c71-193e-4bfa-a394-3341c67d5bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:32.724586 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0252c71-193e-4bfa-a394-3341c67d5bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:32.746043 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59cac9-d236-4866-ab19-692c14f32efe admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:32.748512 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7d59cac9-d236-4866-ab19-692c14f32efe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:32.754762 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-c0252c71-193e-4bfa-a394-3341c67d5bd7 None None] MFA Rules not processed for user `497711da8c3146d6888a19e07e9b1cae`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:44:32.757087 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c0252c71-193e-4bfa-a394-3341c67d5bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:32.774948 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:32.839964 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59cac9-d236-4866-ab19-692c14f32efe admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:32.849114 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1335/2709] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:44:32 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:44:32.854014 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6fce1e-a650-4e94-beff-33985c2519cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:32.854753 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fce1e-a650-4e94-beff-33985c2519cd service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:32.855036 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fce1e-a650-4e94-beff-33985c2519cd service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:32.855309 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fce1e-a650-4e94-beff-33985c2519cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:32.857322 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ed6fce1e-a650-4e94-beff-33985c2519cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:32.862773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0252c71-193e-4bfa-a394-3341c67d5bd7 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:32.864409 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c0252c71-193e-4bfa-a394-3341c67d5bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:32.870870 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6fce1e-a650-4e94-beff-33985c2519cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:32.873520 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6fce1e-a650-4e94-beff-33985c2519cd service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:32.884178 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1375/2710] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:44:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:44:32.928167 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0252c71-193e-4bfa-a394-3341c67d5bd7 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:32.933080 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5744d608-6055-4183-a08e-7ae71500123b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:32.933841 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744d608-6055-4183-a08e-7ae71500123b service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:32.934132 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744d608-6055-4183-a08e-7ae71500123b service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:32.934443 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744d608-6055-4183-a08e-7ae71500123b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:32.946204 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5744d608-6055-4183-a08e-7ae71500123b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:32.947841 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5744d608-6055-4183-a08e-7ae71500123b service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:32.958217 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1336/2711] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:44:32 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:44:33.034481 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6fce1e-a650-4e94-beff-33985c2519cd service nova] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:33.045425 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6fce1e-a650-4e94-beff-33985c2519cd service nova] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:33.049295 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7814d939-27ed-46d6-8d89-8c3f282235d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:33.049959 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814d939-27ed-46d6-8d89-8c3f282235d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:33.050229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814d939-27ed-46d6-8d89-8c3f282235d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:33.050536 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814d939-27ed-46d6-8d89-8c3f282235d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:33.117092 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7814d939-27ed-46d6-8d89-8c3f282235d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:33.118677 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7814d939-27ed-46d6-8d89-8c3f282235d3 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:33.128717 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1376/2712] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:44:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:44:33.424957 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5744d608-6055-4183-a08e-7ae71500123b service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:33.504008 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5744d608-6055-4183-a08e-7ae71500123b service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:33.511031 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eedce7c-8745-43cf-b6ee-c56d25339760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:33.512497 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eedce7c-8745-43cf-b6ee-c56d25339760 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:33.512972 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eedce7c-8745-43cf-b6ee-c56d25339760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:33.513588 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eedce7c-8745-43cf-b6ee-c56d25339760 admin admin] PATH_INFO: `/v3/users/0fdd4c26592c417ca5bd739739708b06` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:33.515645 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eedce7c-8745-43cf-b6ee-c56d25339760 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fdd4c26592c417ca5bd739739708b06)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:33.518014 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eedce7c-8745-43cf-b6ee-c56d25339760 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:33.581479 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0eedce7c-8745-43cf-b6ee-c56d25339760 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fdd4c26592c417ca5bd739739708b06'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:33.581479 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0eedce7c-8745-43cf-b6ee-c56d25339760 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fdd4c26592c417ca5bd739739708b06'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:33.586719 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:33.597259 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0eedce7c-8745-43cf-b6ee-c56d25339760 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fdd4c26592c417ca5bd739739708b06'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:33.604603 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0eedce7c-8745-43cf-b6ee-c56d25339760 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fdd4c26592c417ca5bd739739708b06'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:33.613017 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0eedce7c-8745-43cf-b6ee-c56d25339760 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fdd4c26592c417ca5bd739739708b06'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:33.619728 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1337/2713] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:44:33 2022] DELETE /identity/v3/users/0fdd4c26592c417ca5bd739739708b06 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:33.624933 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f60b6-917a-4295-846b-fb39bcc4a33b None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:33.625218 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f60b6-917a-4295-846b-fb39bcc4a33b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:33.625497 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f60b6-917a-4295-846b-fb39bcc4a33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:33.643393 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-436f60b6-917a-4295-846b-fb39bcc4a33b None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:44:33.644617 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-436f60b6-917a-4295-846b-fb39bcc4a33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:33.719173 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1377/2714] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:44:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:44:33.729418 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eedce7c-8745-43cf-b6ee-c56d25339760 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:33.730987 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0eedce7c-8745-43cf-b6ee-c56d25339760 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:33.801116 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eedce7c-8745-43cf-b6ee-c56d25339760 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:33.807340 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c50a200-c76f-4e31-84e2-c27ec56c5609 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:33.808452 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c50a200-c76f-4e31-84e2-c27ec56c5609 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:33.808935 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c50a200-c76f-4e31-84e2-c27ec56c5609 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:33.809436 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c50a200-c76f-4e31-84e2-c27ec56c5609 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:33.826998 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c50a200-c76f-4e31-84e2-c27ec56c5609 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:33.830072 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c50a200-c76f-4e31-84e2-c27ec56c5609 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:33.845284 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1338/2715] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:44:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:44:34.142691 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436f60b6-917a-4295-846b-fb39bcc4a33b None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:34.145040 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-436f60b6-917a-4295-846b-fb39bcc4a33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:34.159524 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436f60b6-917a-4295-846b-fb39bcc4a33b None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:34.165521 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29642a0-d501-43cd-9b8d-8bce2fe09cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:34.166477 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29642a0-d501-43cd-9b8d-8bce2fe09cd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:34.166767 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29642a0-d501-43cd-9b8d-8bce2fe09cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:34.167067 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29642a0-d501-43cd-9b8d-8bce2fe09cd5 admin admin] PATH_INFO: `/v3/projects/77190064c2bc49808a1a694da39903f1` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:34.168331 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29642a0-d501-43cd-9b8d-8bce2fe09cd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77190064c2bc49808a1a694da39903f1)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:34.170379 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29642a0-d501-43cd-9b8d-8bce2fe09cd5 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:34.182584 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:34.224777 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e29642a0-d501-43cd-9b8d-8bce2fe09cd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77190064c2bc49808a1a694da39903f1', 'request_id': 'req-e29642a0-d501-43cd-9b8d-8bce2fe09cd5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:34.232218 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e29642a0-d501-43cd-9b8d-8bce2fe09cd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77190064c2bc49808a1a694da39903f1', 'request_id': 'req-e29642a0-d501-43cd-9b8d-8bce2fe09cd5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:34.238405 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e29642a0-d501-43cd-9b8d-8bce2fe09cd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77190064c2bc49808a1a694da39903f1', 'request_id': 'req-e29642a0-d501-43cd-9b8d-8bce2fe09cd5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:34.253792 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e29642a0-d501-43cd-9b8d-8bce2fe09cd5 admin admin] The token cache is being invalidate because project 77190064c2bc49808a1a694da39903f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:44:34.254052 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e29642a0-d501-43cd-9b8d-8bce2fe09cd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:34.254806 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e29642a0-d501-43cd-9b8d-8bce2fe09cd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:34.256478 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1378/2716] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:44:34 2022] DELETE /identity/v3/projects/77190064c2bc49808a1a694da39903f1 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:34.260617 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c50a200-c76f-4e31-84e2-c27ec56c5609 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:34.270637 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c50a200-c76f-4e31-84e2-c27ec56c5609 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:34.336345 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eec304-4033-4338-98f6-2b6caae2266e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:34.337237 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eec304-4033-4338-98f6-2b6caae2266e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:34.337504 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eec304-4033-4338-98f6-2b6caae2266e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:34.337788 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eec304-4033-4338-98f6-2b6caae2266e admin admin] PATH_INFO: `/v3/users/497711da8c3146d6888a19e07e9b1cae` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:34.339044 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eec304-4033-4338-98f6-2b6caae2266e admin admin] RBAC: Authorizing `identity:delete_user(user_id=497711da8c3146d6888a19e07e9b1cae)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:34.341016 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eec304-4033-4338-98f6-2b6caae2266e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:34.390066 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-62eec304-4033-4338-98f6-2b6caae2266e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '497711da8c3146d6888a19e07e9b1cae'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:34.390664 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-62eec304-4033-4338-98f6-2b6caae2266e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '497711da8c3146d6888a19e07e9b1cae'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:34.403337 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-62eec304-4033-4338-98f6-2b6caae2266e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '497711da8c3146d6888a19e07e9b1cae'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:34.410621 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-62eec304-4033-4338-98f6-2b6caae2266e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '497711da8c3146d6888a19e07e9b1cae'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:34.412263 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:34.425065 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-62eec304-4033-4338-98f6-2b6caae2266e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '497711da8c3146d6888a19e07e9b1cae'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:34.431742 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1339/2717] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:44:34 2022] DELETE /identity/v3/users/497711da8c3146d6888a19e07e9b1cae => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:34.749894 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29642a0-d501-43cd-9b8d-8bce2fe09cd5 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:34.751756 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e29642a0-d501-43cd-9b8d-8bce2fe09cd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:34.785044 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29642a0-d501-43cd-9b8d-8bce2fe09cd5 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:34.850740 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007b1e0f-7af5-4611-82e6-05b4bf48b8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:34.851741 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b1e0f-7af5-4611-82e6-05b4bf48b8f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:34.852024 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b1e0f-7af5-4611-82e6-05b4bf48b8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:34.852307 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b1e0f-7af5-4611-82e6-05b4bf48b8f3 admin admin] PATH_INFO: `/v3/projects/a5a8a5cab0d24e7884d751c46a5b62a5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:34.853594 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007b1e0f-7af5-4611-82e6-05b4bf48b8f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5a8a5cab0d24e7884d751c46a5b62a5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:34.855769 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007b1e0f-7af5-4611-82e6-05b4bf48b8f3 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:34.910306 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-007b1e0f-7af5-4611-82e6-05b4bf48b8f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5a8a5cab0d24e7884d751c46a5b62a5', 'request_id': 'req-007b1e0f-7af5-4611-82e6-05b4bf48b8f3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:34.917068 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-007b1e0f-7af5-4611-82e6-05b4bf48b8f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5a8a5cab0d24e7884d751c46a5b62a5', 'request_id': 'req-007b1e0f-7af5-4611-82e6-05b4bf48b8f3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:34.925587 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-007b1e0f-7af5-4611-82e6-05b4bf48b8f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5a8a5cab0d24e7884d751c46a5b62a5', 'request_id': 'req-007b1e0f-7af5-4611-82e6-05b4bf48b8f3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:34.943119 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-007b1e0f-7af5-4611-82e6-05b4bf48b8f3 admin admin] The token cache is being invalidate because project a5a8a5cab0d24e7884d751c46a5b62a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:44:34.943119 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-007b1e0f-7af5-4611-82e6-05b4bf48b8f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:34.944345 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-007b1e0f-7af5-4611-82e6-05b4bf48b8f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:34.946598 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1379/2718] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:44:34 2022] DELETE /identity/v3/projects/a5a8a5cab0d24e7884d751c46a5b62a5 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:34.956916 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c941d11-9729-46f2-abc6-7dbd2b46e86e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:34.956916 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c941d11-9729-46f2-abc6-7dbd2b46e86e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:34.956916 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c941d11-9729-46f2-abc6-7dbd2b46e86e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:34.974782 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-4c941d11-9729-46f2-abc6-7dbd2b46e86e None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:44:34.976044 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4c941d11-9729-46f2-abc6-7dbd2b46e86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:35.053933 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1340/2719] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:44:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:44:35.062729 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c72a7-c609-4410-a977-54bc8422641b None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:35.063155 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c72a7-c609-4410-a977-54bc8422641b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:35.063622 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c72a7-c609-4410-a977-54bc8422641b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:35.102750 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-1a9c72a7-c609-4410-a977-54bc8422641b None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:44:35.104021 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1a9c72a7-c609-4410-a977-54bc8422641b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:35.114179 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1380/2720] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:44:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:44:35.119559 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c941d11-9729-46f2-abc6-7dbd2b46e86e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:35.130310 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c941d11-9729-46f2-abc6-7dbd2b46e86e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:35.136634 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76212e49-0d5b-4882-a786-0a3cf0f41514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:35.137326 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76212e49-0d5b-4882-a786-0a3cf0f41514 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:35.137607 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76212e49-0d5b-4882-a786-0a3cf0f41514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:35.138015 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76212e49-0d5b-4882-a786-0a3cf0f41514 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:35.139123 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76212e49-0d5b-4882-a786-0a3cf0f41514 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:44:35.139407 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76212e49-0d5b-4882-a786-0a3cf0f41514 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:35.141024 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76212e49-0d5b-4882-a786-0a3cf0f41514 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:35.155703 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1341/2721] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:44:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:44:35.159295 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c72a7-c609-4410-a977-54bc8422641b None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:35.169360 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c72a7-c609-4410-a977-54bc8422641b None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:35.174971 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0901d4f9-5982-4592-87e6-c68bf754116e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:35.175988 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0901d4f9-5982-4592-87e6-c68bf754116e admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:35.176414 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0901d4f9-5982-4592-87e6-c68bf754116e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:35.176888 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0901d4f9-5982-4592-87e6-c68bf754116e admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:35.177837 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0901d4f9-5982-4592-87e6-c68bf754116e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:35.179960 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0901d4f9-5982-4592-87e6-c68bf754116e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:35.209125 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:35.222580 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1381/2722] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:44:35 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:44:35.224590 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76212e49-0d5b-4882-a786-0a3cf0f41514 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:35.235058 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76212e49-0d5b-4882-a786-0a3cf0f41514 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:35.301278 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883cdedd-ec10-4630-9fac-6baaaab594ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:35.301978 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883cdedd-ec10-4630-9fac-6baaaab594ff admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:35.302292 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883cdedd-ec10-4630-9fac-6baaaab594ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:35.302574 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883cdedd-ec10-4630-9fac-6baaaab594ff admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:35.303284 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883cdedd-ec10-4630-9fac-6baaaab594ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:35.305292 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883cdedd-ec10-4630-9fac-6baaaab594ff admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:35.328998 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1342/2723] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:44:35 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:44:35.333197 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0901d4f9-5982-4592-87e6-c68bf754116e admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:35.344292 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0901d4f9-5982-4592-87e6-c68bf754116e admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:35.349745 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b3bdd1-a8ea-4667-8c67-7b618e25d5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:35.350538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b3bdd1-a8ea-4667-8c67-7b618e25d5b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:35.350882 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b3bdd1-a8ea-4667-8c67-7b618e25d5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:35.351800 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b3bdd1-a8ea-4667-8c67-7b618e25d5b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:35.352006 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b3bdd1-a8ea-4667-8c67-7b618e25d5b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:35.353674 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b3bdd1-a8ea-4667-8c67-7b618e25d5b4 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:35.364810 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1382/2724] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:44:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:44:35.369067 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883cdedd-ec10-4630-9fac-6baaaab594ff admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:35.380467 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883cdedd-ec10-4630-9fac-6baaaab594ff admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:35.385359 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce62e45-ca9c-4f6f-82f9-35f9db0d3773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:35.386113 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce62e45-ca9c-4f6f-82f9-35f9db0d3773 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:35.386501 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce62e45-ca9c-4f6f-82f9-35f9db0d3773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:35.386811 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce62e45-ca9c-4f6f-82f9-35f9db0d3773 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:35.387548 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce62e45-ca9c-4f6f-82f9-35f9db0d3773 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:35.389212 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce62e45-ca9c-4f6f-82f9-35f9db0d3773 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:35.399561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1343/2725] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:44:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:44:35.403768 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b3bdd1-a8ea-4667-8c67-7b618e25d5b4 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:35.415321 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b3bdd1-a8ea-4667-8c67-7b618e25d5b4 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:35.420811 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff5902-7dd4-46a2-bf2a-b9649d4285e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:35.421722 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff5902-7dd4-46a2-bf2a-b9649d4285e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:35.422073 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff5902-7dd4-46a2-bf2a-b9649d4285e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:35.422488 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff5902-7dd4-46a2-bf2a-b9649d4285e6 admin admin] PATH_INFO: `/v3/projects/7eae7fe4575745e680dc1b9367941005/users/8fa3778bf9f94af7ae5c51dfcf054250/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:35.438804 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ff5902-7dd4-46a2-bf2a-b9649d4285e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7eae7fe4575745e680dc1b9367941005, user_id=8fa3778bf9f94af7ae5c51dfcf054250, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:35.441619 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ff5902-7dd4-46a2-bf2a-b9649d4285e6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:35.457543 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1383/2726] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:44:35 2022] PUT /identity/v3/projects/7eae7fe4575745e680dc1b9367941005/users/8fa3778bf9f94af7ae5c51dfcf054250/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:35.470077 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a320c-8516-4c23-9910-674e9df634ae None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:35.470465 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a320c-8516-4c23-9910-674e9df634ae None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:35.470693 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a320c-8516-4c23-9910-674e9df634ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:35.502464 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-722a320c-8516-4c23-9910-674e9df634ae None None] MFA Rules not processed for user `8fa3778bf9f94af7ae5c51dfcf054250`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:44:35.503749 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-722a320c-8516-4c23-9910-674e9df634ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:35.555807 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:35.581916 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1344/2727] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:44:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:44:35.586709 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff5902-7dd4-46a2-bf2a-b9649d4285e6 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:35.657695 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff5902-7dd4-46a2-bf2a-b9649d4285e6 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:35.662651 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fa9daf-54d5-48d2-a22f-9413a4dcc3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:35.663333 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa9daf-54d5-48d2-a22f-9413a4dcc3b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:35.663612 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa9daf-54d5-48d2-a22f-9413a4dcc3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:35.663878 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa9daf-54d5-48d2-a22f-9413a4dcc3b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:35.664599 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fa9daf-54d5-48d2-a22f-9413a4dcc3b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:35.666561 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fa9daf-54d5-48d2-a22f-9413a4dcc3b0 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:35.697017 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1384/2728] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:44:35 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:44:35.700979 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a320c-8516-4c23-9910-674e9df634ae None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:35.713122 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a320c-8516-4c23-9910-674e9df634ae None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:35.775816 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a1e0f2-d6b2-47c5-8304-ff4caec32dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:35.776465 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a1e0f2-d6b2-47c5-8304-ff4caec32dda admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:35.776787 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a1e0f2-d6b2-47c5-8304-ff4caec32dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:35.777091 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a1e0f2-d6b2-47c5-8304-ff4caec32dda admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:35.777854 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a1e0f2-d6b2-47c5-8304-ff4caec32dda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:35.780305 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a1e0f2-d6b2-47c5-8304-ff4caec32dda admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:35.806920 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1345/2729] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:44:35 2022] POST /identity/v3/users => generated 299 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:44:35.811042 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fa9daf-54d5-48d2-a22f-9413a4dcc3b0 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:35.822524 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fa9daf-54d5-48d2-a22f-9413a4dcc3b0 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:35.827980 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e01198-9a3c-4c19-af9e-a722bf2a7527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:35.828660 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e01198-9a3c-4c19-af9e-a722bf2a7527 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:35.828927 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e01198-9a3c-4c19-af9e-a722bf2a7527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:35.829205 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e01198-9a3c-4c19-af9e-a722bf2a7527 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:35.829948 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e01198-9a3c-4c19-af9e-a722bf2a7527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:35.831620 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e01198-9a3c-4c19-af9e-a722bf2a7527 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:35.843240 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1385/2730] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:44:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:44:35.847892 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a1e0f2-d6b2-47c5-8304-ff4caec32dda admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:35.859777 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a1e0f2-d6b2-47c5-8304-ff4caec32dda admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:35.864701 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128b192-d6fa-449e-93c6-8a359291e4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:35.865680 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128b192-d6fa-449e-93c6-8a359291e4a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:35.866030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128b192-d6fa-449e-93c6-8a359291e4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:35.866433 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128b192-d6fa-449e-93c6-8a359291e4a7 admin admin] PATH_INFO: `/v3/projects/3f45caa804af499a8393266fa1ca440f/users/299dc8eac9d84a1880e5c6f3fb32f9c7/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:35.882336 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1128b192-d6fa-449e-93c6-8a359291e4a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f45caa804af499a8393266fa1ca440f, user_id=299dc8eac9d84a1880e5c6f3fb32f9c7, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:35.885074 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1128b192-d6fa-449e-93c6-8a359291e4a7 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:35.899612 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1346/2731] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:44:35 2022] PUT /identity/v3/projects/3f45caa804af499a8393266fa1ca440f/users/299dc8eac9d84a1880e5c6f3fb32f9c7/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:35.903743 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e01198-9a3c-4c19-af9e-a722bf2a7527 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:35.913830 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e01198-9a3c-4c19-af9e-a722bf2a7527 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:35.970133 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b70e632-bc55-4bfe-aad5-2d8b5ef1535d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:35.970769 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b70e632-bc55-4bfe-aad5-2d8b5ef1535d admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:35.971046 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b70e632-bc55-4bfe-aad5-2d8b5ef1535d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:35.971332 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b70e632-bc55-4bfe-aad5-2d8b5ef1535d admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:35.972174 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b70e632-bc55-4bfe-aad5-2d8b5ef1535d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:35.973708 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b70e632-bc55-4bfe-aad5-2d8b5ef1535d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:35.985163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1386/2732] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:44:35 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:44:35.991328 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128b192-d6fa-449e-93c6-8a359291e4a7 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:36.002899 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128b192-d6fa-449e-93c6-8a359291e4a7 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:36.007540 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee80681-c153-4890-884f-f4a5e8c57c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:36.008441 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee80681-c153-4890-884f-f4a5e8c57c1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:36.008716 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee80681-c153-4890-884f-f4a5e8c57c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:36.008999 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee80681-c153-4890-884f-f4a5e8c57c1e admin admin] PATH_INFO: `/v3/projects/3f45caa804af499a8393266fa1ca440f/users/299dc8eac9d84a1880e5c6f3fb32f9c7/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:36.010939 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee80681-c153-4890-884f-f4a5e8c57c1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f45caa804af499a8393266fa1ca440f, user_id=299dc8eac9d84a1880e5c6f3fb32f9c7, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:36.013933 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee80681-c153-4890-884f-f4a5e8c57c1e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:36.026104 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1347/2733] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:44:35 2022] PUT /identity/v3/projects/3f45caa804af499a8393266fa1ca440f/users/299dc8eac9d84a1880e5c6f3fb32f9c7/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:36.036128 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9154c2d8-d4d4-484e-b24d-b94e825866f8 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:36.036420 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9154c2d8-d4d4-484e-b24d-b94e825866f8 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:36.036702 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9154c2d8-d4d4-484e-b24d-b94e825866f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:36.065454 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9154c2d8-d4d4-484e-b24d-b94e825866f8 None None] MFA Rules not processed for user `299dc8eac9d84a1880e5c6f3fb32f9c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:44:36.066782 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9154c2d8-d4d4-484e-b24d-b94e825866f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:36.091528 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:36.144544 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1387/2734] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:44:36 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:44:36.153081 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee80681-c153-4890-884f-f4a5e8c57c1e admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:36.155275 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8ee80681-c153-4890-884f-f4a5e8c57c1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:36.220859 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee80681-c153-4890-884f-f4a5e8c57c1e admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:36.224582 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba5d2d-52b1-4eaa-baa0-fde4c33582e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:36.225267 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba5d2d-52b1-4eaa-baa0-fde4c33582e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:36.225548 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba5d2d-52b1-4eaa-baa0-fde4c33582e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:36.225850 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba5d2d-52b1-4eaa-baa0-fde4c33582e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:36.282343 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:36.286643 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ba5d2d-52b1-4eaa-baa0-fde4c33582e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:36.289395 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ba5d2d-52b1-4eaa-baa0-fde4c33582e9 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:36.300556 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1348/2735] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:44:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:44:40.640927 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9154c2d8-d4d4-484e-b24d-b94e825866f8 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:40.643128 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9154c2d8-d4d4-484e-b24d-b94e825866f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:40.657809 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9154c2d8-d4d4-484e-b24d-b94e825866f8 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:40.661944 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14ab23-d23c-4018-935e-64706d816e00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:40.662754 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14ab23-d23c-4018-935e-64706d816e00 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:40.663195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14ab23-d23c-4018-935e-64706d816e00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:40.663670 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14ab23-d23c-4018-935e-64706d816e00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:40.676944 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca14ab23-d23c-4018-935e-64706d816e00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:40.678449 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca14ab23-d23c-4018-935e-64706d816e00 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:40.688996 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1388/2736] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:44:40 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:44:45.615832 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba5d2d-52b1-4eaa-baa0-fde4c33582e9 service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:45.691950 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba5d2d-52b1-4eaa-baa0-fde4c33582e9 service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:45.697904 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade97893-ba36-4685-b572-15dcbc36d284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:45.698855 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade97893-ba36-4685-b572-15dcbc36d284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:45.699138 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade97893-ba36-4685-b572-15dcbc36d284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:45.699412 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade97893-ba36-4685-b572-15dcbc36d284 admin admin] PATH_INFO: `/v3/users/8fa3778bf9f94af7ae5c51dfcf054250` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:45.700688 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade97893-ba36-4685-b572-15dcbc36d284 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fa3778bf9f94af7ae5c51dfcf054250)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:45.702734 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade97893-ba36-4685-b572-15dcbc36d284 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:45.760175 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ade97893-ba36-4685-b572-15dcbc36d284 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fa3778bf9f94af7ae5c51dfcf054250'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:45.760903 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ade97893-ba36-4685-b572-15dcbc36d284 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fa3778bf9f94af7ae5c51dfcf054250'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:45.774629 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ade97893-ba36-4685-b572-15dcbc36d284 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fa3778bf9f94af7ae5c51dfcf054250'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:45.784856 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ade97893-ba36-4685-b572-15dcbc36d284 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fa3778bf9f94af7ae5c51dfcf054250'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:45.795153 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ade97893-ba36-4685-b572-15dcbc36d284 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fa3778bf9f94af7ae5c51dfcf054250'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:45.802714 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1349/2737] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:44:45 2022] DELETE /identity/v3/users/8fa3778bf9f94af7ae5c51dfcf054250 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:45.807908 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1a1239-d9ae-4a07-b3bc-7b339365be1b None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:45.808638 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1a1239-d9ae-4a07-b3bc-7b339365be1b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:45.808800 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1a1239-d9ae-4a07-b3bc-7b339365be1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:45.829396 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-3a1a1239-d9ae-4a07-b3bc-7b339365be1b None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:44:45.830699 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3a1a1239-d9ae-4a07-b3bc-7b339365be1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:45.894821 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1389/2738] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:44:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:44:45.904849 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade97893-ba36-4685-b572-15dcbc36d284 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:45.906741 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ade97893-ba36-4685-b572-15dcbc36d284 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:45.970359 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade97893-ba36-4685-b572-15dcbc36d284 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:45.974574 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f1e60-c5c4-4c3e-8f75-8e22259f3410 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:45.975235 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f1e60-c5c4-4c3e-8f75-8e22259f3410 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:45.975516 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f1e60-c5c4-4c3e-8f75-8e22259f3410 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:45.975793 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f1e60-c5c4-4c3e-8f75-8e22259f3410 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:45.986387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f1e60-c5c4-4c3e-8f75-8e22259f3410 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:45.987878 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f1e60-c5c4-4c3e-8f75-8e22259f3410 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:45.998260 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1350/2739] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:44:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:44:46.310659 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1a1239-d9ae-4a07-b3bc-7b339365be1b None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:46.313191 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3a1a1239-d9ae-4a07-b3bc-7b339365be1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:46.340910 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1a1239-d9ae-4a07-b3bc-7b339365be1b None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:46.349056 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc029fb9-c47f-4430-a620-cdfbb797f2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:46.350435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc029fb9-c47f-4430-a620-cdfbb797f2d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:46.350725 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc029fb9-c47f-4430-a620-cdfbb797f2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:46.351049 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc029fb9-c47f-4430-a620-cdfbb797f2d7 admin admin] PATH_INFO: `/v3/projects/7eae7fe4575745e680dc1b9367941005` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:46.353280 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc029fb9-c47f-4430-a620-cdfbb797f2d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7eae7fe4575745e680dc1b9367941005)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:46.356305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc029fb9-c47f-4430-a620-cdfbb797f2d7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:46.418889 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cc029fb9-c47f-4430-a620-cdfbb797f2d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7eae7fe4575745e680dc1b9367941005', 'request_id': 'req-cc029fb9-c47f-4430-a620-cdfbb797f2d7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:46.426576 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cc029fb9-c47f-4430-a620-cdfbb797f2d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7eae7fe4575745e680dc1b9367941005', 'request_id': 'req-cc029fb9-c47f-4430-a620-cdfbb797f2d7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:46.432785 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cc029fb9-c47f-4430-a620-cdfbb797f2d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7eae7fe4575745e680dc1b9367941005', 'request_id': 'req-cc029fb9-c47f-4430-a620-cdfbb797f2d7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:46.447772 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cc029fb9-c47f-4430-a620-cdfbb797f2d7 admin admin] The token cache is being invalidate because project 7eae7fe4575745e680dc1b9367941005 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:44:46.448063 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cc029fb9-c47f-4430-a620-cdfbb797f2d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:46.448753 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cc029fb9-c47f-4430-a620-cdfbb797f2d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:46.450414 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1390/2740] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:44:46 2022] DELETE /identity/v3/projects/7eae7fe4575745e680dc1b9367941005 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:46.456335 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f1e60-c5c4-4c3e-8f75-8e22259f3410 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:46.470518 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f1e60-c5c4-4c3e-8f75-8e22259f3410 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:46.489201 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:46.546604 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e13e6-e3bf-483f-8ab4-288e87626f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:46.548202 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e13e6-e3bf-483f-8ab4-288e87626f00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:46.548489 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e13e6-e3bf-483f-8ab4-288e87626f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:46.548764 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e13e6-e3bf-483f-8ab4-288e87626f00 admin admin] PATH_INFO: `/v3/users/299dc8eac9d84a1880e5c6f3fb32f9c7` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:46.550032 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858e13e6-e3bf-483f-8ab4-288e87626f00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=299dc8eac9d84a1880e5c6f3fb32f9c7)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:46.552024 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858e13e6-e3bf-483f-8ab4-288e87626f00 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:46.604754 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-858e13e6-e3bf-483f-8ab4-288e87626f00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '299dc8eac9d84a1880e5c6f3fb32f9c7'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:46.605382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-858e13e6-e3bf-483f-8ab4-288e87626f00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '299dc8eac9d84a1880e5c6f3fb32f9c7'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:46.629691 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-858e13e6-e3bf-483f-8ab4-288e87626f00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '299dc8eac9d84a1880e5c6f3fb32f9c7'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:46.637583 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-858e13e6-e3bf-483f-8ab4-288e87626f00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '299dc8eac9d84a1880e5c6f3fb32f9c7'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:46.651309 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-858e13e6-e3bf-483f-8ab4-288e87626f00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '299dc8eac9d84a1880e5c6f3fb32f9c7'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:46.658617 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1351/2741] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:44:46 2022] DELETE /identity/v3/users/299dc8eac9d84a1880e5c6f3fb32f9c7 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:46.975215 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc029fb9-c47f-4430-a620-cdfbb797f2d7 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:46.976629 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cc029fb9-c47f-4430-a620-cdfbb797f2d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:46.990614 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc029fb9-c47f-4430-a620-cdfbb797f2d7 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:47.022365 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:47.055146 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a3a2f-5905-4688-93bd-e61ea68790fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:47.056037 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a3a2f-5905-4688-93bd-e61ea68790fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:47.056317 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a3a2f-5905-4688-93bd-e61ea68790fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:47.056595 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a3a2f-5905-4688-93bd-e61ea68790fa admin admin] PATH_INFO: `/v3/projects/3f45caa804af499a8393266fa1ca440f` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:47.057841 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522a3a2f-5905-4688-93bd-e61ea68790fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f45caa804af499a8393266fa1ca440f)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:47.059813 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522a3a2f-5905-4688-93bd-e61ea68790fa admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:47.112394 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-522a3a2f-5905-4688-93bd-e61ea68790fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f45caa804af499a8393266fa1ca440f', 'request_id': 'req-522a3a2f-5905-4688-93bd-e61ea68790fa'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:47.118850 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-522a3a2f-5905-4688-93bd-e61ea68790fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f45caa804af499a8393266fa1ca440f', 'request_id': 'req-522a3a2f-5905-4688-93bd-e61ea68790fa'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:47.124468 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-522a3a2f-5905-4688-93bd-e61ea68790fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f45caa804af499a8393266fa1ca440f', 'request_id': 'req-522a3a2f-5905-4688-93bd-e61ea68790fa'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:47.139553 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-522a3a2f-5905-4688-93bd-e61ea68790fa admin admin] The token cache is being invalidate because project 3f45caa804af499a8393266fa1ca440f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:44:47.139838 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-522a3a2f-5905-4688-93bd-e61ea68790fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:47.140444 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-522a3a2f-5905-4688-93bd-e61ea68790fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:47.142257 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1391/2742] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:44:46 2022] DELETE /identity/v3/projects/3f45caa804af499a8393266fa1ca440f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:44:47.150258 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e68da75-31c5-4a46-aeda-5f5fa124f86c None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:47.150701 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e68da75-31c5-4a46-aeda-5f5fa124f86c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:47.151269 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e68da75-31c5-4a46-aeda-5f5fa124f86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:47.169915 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-9e68da75-31c5-4a46-aeda-5f5fa124f86c None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:44:47.171228 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9e68da75-31c5-4a46-aeda-5f5fa124f86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:47.234151 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1352/2743] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:44:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:44:47.245509 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72601b6b-f322-44e7-bd72-eb88d330ff80 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:47.245898 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72601b6b-f322-44e7-bd72-eb88d330ff80 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:47.246286 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72601b6b-f322-44e7-bd72-eb88d330ff80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:47.268033 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-72601b6b-f322-44e7-bd72-eb88d330ff80 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:44:47.269258 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-72601b6b-f322-44e7-bd72-eb88d330ff80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:47.277649 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1392/2744] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:44:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:44:47.284040 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e68da75-31c5-4a46-aeda-5f5fa124f86c None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:47.294301 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e68da75-31c5-4a46-aeda-5f5fa124f86c None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:47.299992 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff4197-107a-4018-9244-23fb0432e73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:47.300671 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff4197-107a-4018-9244-23fb0432e73a admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:47.300988 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff4197-107a-4018-9244-23fb0432e73a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:47.301684 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff4197-107a-4018-9244-23fb0432e73a admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:47.302006 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ff4197-107a-4018-9244-23fb0432e73a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:44:47.302382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ff4197-107a-4018-9244-23fb0432e73a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:47.303976 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ff4197-107a-4018-9244-23fb0432e73a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:47.317260 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1353/2745] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:44:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:44:47.323792 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72601b6b-f322-44e7-bd72-eb88d330ff80 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:47.334601 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72601b6b-f322-44e7-bd72-eb88d330ff80 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:47.339529 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d558c-9d5b-4790-ab3e-3a763d30a690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:47.340202 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d558c-9d5b-4790-ab3e-3a763d30a690 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:47.340484 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d558c-9d5b-4790-ab3e-3a763d30a690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:47.340756 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d558c-9d5b-4790-ab3e-3a763d30a690 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:47.341463 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56d558c-9d5b-4790-ab3e-3a763d30a690 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:47.343434 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56d558c-9d5b-4790-ab3e-3a763d30a690 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:47.372724 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1393/2746] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:44:47 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:44:47.377558 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff4197-107a-4018-9244-23fb0432e73a admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:47.388582 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff4197-107a-4018-9244-23fb0432e73a admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:47.412087 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:47.457552 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9286dc14-e028-42cc-9661-e531db9e5c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:47.458261 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286dc14-e028-42cc-9661-e531db9e5c6a admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:47.458583 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286dc14-e028-42cc-9661-e531db9e5c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:47.458856 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286dc14-e028-42cc-9661-e531db9e5c6a admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:47.459555 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9286dc14-e028-42cc-9661-e531db9e5c6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:47.461495 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9286dc14-e028-42cc-9661-e531db9e5c6a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:47.485273 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1354/2747] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:44:47 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:44:47.489865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d558c-9d5b-4790-ab3e-3a763d30a690 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:47.503526 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d558c-9d5b-4790-ab3e-3a763d30a690 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:47.509125 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32332a7-c95f-4cd4-ab62-afe13a9c98b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:47.509766 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32332a7-c95f-4cd4-ab62-afe13a9c98b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:47.510031 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32332a7-c95f-4cd4-ab62-afe13a9c98b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:47.510352 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32332a7-c95f-4cd4-ab62-afe13a9c98b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:47.511085 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32332a7-c95f-4cd4-ab62-afe13a9c98b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:47.512733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32332a7-c95f-4cd4-ab62-afe13a9c98b2 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:47.525636 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1394/2748] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:44:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:44:47.529965 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9286dc14-e028-42cc-9661-e531db9e5c6a admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:47.540925 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9286dc14-e028-42cc-9661-e531db9e5c6a admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:47.546005 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f50c3-a5c6-4fef-9a72-cdb3016d58f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:47.546726 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f50c3-a5c6-4fef-9a72-cdb3016d58f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:47.547045 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f50c3-a5c6-4fef-9a72-cdb3016d58f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:47.547320 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f50c3-a5c6-4fef-9a72-cdb3016d58f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:47.548059 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178f50c3-a5c6-4fef-9a72-cdb3016d58f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:47.549677 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178f50c3-a5c6-4fef-9a72-cdb3016d58f5 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:47.560456 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1355/2749] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:44:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:44:47.565418 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32332a7-c95f-4cd4-ab62-afe13a9c98b2 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:47.577411 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32332a7-c95f-4cd4-ab62-afe13a9c98b2 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:47.584314 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1496af-5d5f-4665-a72d-dd38cd448950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:47.585226 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1496af-5d5f-4665-a72d-dd38cd448950 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:47.585515 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1496af-5d5f-4665-a72d-dd38cd448950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:47.585783 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1496af-5d5f-4665-a72d-dd38cd448950 admin admin] PATH_INFO: `/v3/projects/7f22ab4110204152a909fa5526c47307/users/be0d403201ef42eda4ffe8d3ad6539dc/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:47.602312 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1496af-5d5f-4665-a72d-dd38cd448950 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f22ab4110204152a909fa5526c47307, user_id=be0d403201ef42eda4ffe8d3ad6539dc, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:47.605075 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1496af-5d5f-4665-a72d-dd38cd448950 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:47.620869 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1395/2750] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:44:47 2022] PUT /identity/v3/projects/7f22ab4110204152a909fa5526c47307/users/be0d403201ef42eda4ffe8d3ad6539dc/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:47.691799 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db9a3a-0810-4a5a-bb8f-848e0129333c None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:47.692399 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db9a3a-0810-4a5a-bb8f-848e0129333c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:47.692399 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db9a3a-0810-4a5a-bb8f-848e0129333c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:47.727965 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b3db9a3a-0810-4a5a-bb8f-848e0129333c None None] MFA Rules not processed for user `be0d403201ef42eda4ffe8d3ad6539dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:44:47.729601 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b3db9a3a-0810-4a5a-bb8f-848e0129333c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:47.805394 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1356/2751] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:44:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:44:47.810551 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1496af-5d5f-4665-a72d-dd38cd448950 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:47.880308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:47.884575 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1496af-5d5f-4665-a72d-dd38cd448950 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:47.891777 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a31045-817c-4f86-8764-d92a43bc2157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:47.892651 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a31045-817c-4f86-8764-d92a43bc2157 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:47.893473 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a31045-817c-4f86-8764-d92a43bc2157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:47.893473 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a31045-817c-4f86-8764-d92a43bc2157 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:47.894432 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a31045-817c-4f86-8764-d92a43bc2157 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:47.896378 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a31045-817c-4f86-8764-d92a43bc2157 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:47.925538 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1396/2752] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:44:47 2022] POST /identity/v3/projects => generated 375 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:44:47.930861 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db9a3a-0810-4a5a-bb8f-848e0129333c None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:47.944961 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db9a3a-0810-4a5a-bb8f-848e0129333c None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:48.010587 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8951e69c-fec6-4e24-9afe-7b52d34c137a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:48.011290 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8951e69c-fec6-4e24-9afe-7b52d34c137a admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:48.011577 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8951e69c-fec6-4e24-9afe-7b52d34c137a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:48.011857 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8951e69c-fec6-4e24-9afe-7b52d34c137a admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:48.012721 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8951e69c-fec6-4e24-9afe-7b52d34c137a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:48.015133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8951e69c-fec6-4e24-9afe-7b52d34c137a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:48.042380 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1357/2753] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:44:47 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:44:48.043696 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a31045-817c-4f86-8764-d92a43bc2157 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:48.055238 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a31045-817c-4f86-8764-d92a43bc2157 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:48.060154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff50cc4e-012a-4939-83bd-7de99a8ec120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:48.060853 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50cc4e-012a-4939-83bd-7de99a8ec120 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:48.061142 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50cc4e-012a-4939-83bd-7de99a8ec120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:48.061422 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50cc4e-012a-4939-83bd-7de99a8ec120 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:48.062167 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff50cc4e-012a-4939-83bd-7de99a8ec120 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:48.064039 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff50cc4e-012a-4939-83bd-7de99a8ec120 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:48.076023 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1397/2754] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 21:44:48.080469 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8951e69c-fec6-4e24-9afe-7b52d34c137a admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:48.091440 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8951e69c-fec6-4e24-9afe-7b52d34c137a admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:48.099579 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b1bb73-1247-49ee-af9d-7ed01fd0bdcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:48.100535 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1bb73-1247-49ee-af9d-7ed01fd0bdcf admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:48.100965 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1bb73-1247-49ee-af9d-7ed01fd0bdcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:48.101427 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1bb73-1247-49ee-af9d-7ed01fd0bdcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:48.102503 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b1bb73-1247-49ee-af9d-7ed01fd0bdcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:48.105013 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b1bb73-1247-49ee-af9d-7ed01fd0bdcf admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:48.117605 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1358/2755] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:44:48.122001 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff50cc4e-012a-4939-83bd-7de99a8ec120 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:48.132912 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff50cc4e-012a-4939-83bd-7de99a8ec120 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:48.138220 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364537e0-4fdd-48c9-9e28-ce7e4d2c33bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:48.139152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364537e0-4fdd-48c9-9e28-ce7e4d2c33bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:48.139443 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364537e0-4fdd-48c9-9e28-ce7e4d2c33bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:48.139710 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364537e0-4fdd-48c9-9e28-ce7e4d2c33bd admin admin] PATH_INFO: `/v3/projects/303be763807c4e27911870f876fa96eb/users/f30312d064514b389032c8c035591ea0/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:48.156384 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364537e0-4fdd-48c9-9e28-ce7e4d2c33bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=303be763807c4e27911870f876fa96eb, user_id=f30312d064514b389032c8c035591ea0, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:48.159066 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364537e0-4fdd-48c9-9e28-ce7e4d2c33bd admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:48.174504 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1398/2756] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:44:48 2022] PUT /identity/v3/projects/303be763807c4e27911870f876fa96eb/users/f30312d064514b389032c8c035591ea0/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:48.185157 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d965d-7fc8-4921-9c9c-d23bf1d320d6 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:48.185447 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d965d-7fc8-4921-9c9c-d23bf1d320d6 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:48.185731 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d965d-7fc8-4921-9c9c-d23bf1d320d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:48.210210 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:48.221799 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-868d965d-7fc8-4921-9c9c-d23bf1d320d6 None None] MFA Rules not processed for user `f30312d064514b389032c8c035591ea0`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:44:48.223194 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-868d965d-7fc8-4921-9c9c-d23bf1d320d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:48.291828 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1359/2757] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:44:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:44:48.297632 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364537e0-4fdd-48c9-9e28-ce7e4d2c33bd admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:48.380733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364537e0-4fdd-48c9-9e28-ce7e4d2c33bd admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:48.389141 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed143d-b909-4e9a-8bc3-6c52bca97dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:48.389945 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed143d-b909-4e9a-8bc3-6c52bca97dc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:48.390212 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed143d-b909-4e9a-8bc3-6c52bca97dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:48.390913 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed143d-b909-4e9a-8bc3-6c52bca97dc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:48.391424 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed143d-b909-4e9a-8bc3-6c52bca97dc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:48.394795 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed143d-b909-4e9a-8bc3-6c52bca97dc7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:48.428514 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1399/2758] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:44:48 2022] POST /identity/v3/projects => generated 377 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:44:48.434114 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d965d-7fc8-4921-9c9c-d23bf1d320d6 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:48.445715 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d965d-7fc8-4921-9c9c-d23bf1d320d6 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:48.504489 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80ec97-c3d8-432e-af43-37ded97312eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:48.505089 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80ec97-c3d8-432e-af43-37ded97312eb admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:48.505370 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80ec97-c3d8-432e-af43-37ded97312eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:48.505643 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80ec97-c3d8-432e-af43-37ded97312eb admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:48.506391 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b80ec97-c3d8-432e-af43-37ded97312eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:48.508324 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b80ec97-c3d8-432e-af43-37ded97312eb admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:48.531087 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1360/2759] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:44:48 2022] POST /identity/v3/users => generated 295 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:44:48.536071 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed143d-b909-4e9a-8bc3-6c52bca97dc7 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:48.548298 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed143d-b909-4e9a-8bc3-6c52bca97dc7 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:48.553908 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e392176-6193-430e-99f5-49178596d391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:48.554619 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e392176-6193-430e-99f5-49178596d391 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:48.554895 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e392176-6193-430e-99f5-49178596d391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:48.555171 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e392176-6193-430e-99f5-49178596d391 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:48.555906 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e392176-6193-430e-99f5-49178596d391 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:48.557519 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e392176-6193-430e-99f5-49178596d391 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:48.569528 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1400/2760] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:44:48.575465 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80ec97-c3d8-432e-af43-37ded97312eb admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:48.586344 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80ec97-c3d8-432e-af43-37ded97312eb admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:48.591602 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77187dbb-66c5-47ec-a532-00d0a402e20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:48.592496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77187dbb-66c5-47ec-a532-00d0a402e20a admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:48.592803 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77187dbb-66c5-47ec-a532-00d0a402e20a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:48.593074 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77187dbb-66c5-47ec-a532-00d0a402e20a admin admin] PATH_INFO: `/v3/projects/98b04880708e44bfb7e3455e1ebb8e7b/users/2b8fddaa59cc4a90ab3fe0123bdee9d4/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:48.608737 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77187dbb-66c5-47ec-a532-00d0a402e20a admin admin] RBAC: Authorizing `identity:create_grant(project_id=98b04880708e44bfb7e3455e1ebb8e7b, user_id=2b8fddaa59cc4a90ab3fe0123bdee9d4, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:48.611730 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77187dbb-66c5-47ec-a532-00d0a402e20a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:48.632914 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1361/2761] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:44:48 2022] PUT /identity/v3/projects/98b04880708e44bfb7e3455e1ebb8e7b/users/2b8fddaa59cc4a90ab3fe0123bdee9d4/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:44:48.639036 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e392176-6193-430e-99f5-49178596d391 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:48.650215 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e392176-6193-430e-99f5-49178596d391 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:48.717230 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265d0c41-fb17-4549-b541-cba0eed235ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:48.717918 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d0c41-fb17-4549-b541-cba0eed235ef admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:48.718203 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d0c41-fb17-4549-b541-cba0eed235ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:48.718555 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d0c41-fb17-4549-b541-cba0eed235ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:48.719873 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265d0c41-fb17-4549-b541-cba0eed235ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:48.721495 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265d0c41-fb17-4549-b541-cba0eed235ef admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:48.732670 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1401/2762] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:44:48.740117 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77187dbb-66c5-47ec-a532-00d0a402e20a admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:48.751252 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77187dbb-66c5-47ec-a532-00d0a402e20a admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:48.757527 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce1b90b-49b3-40d9-9b52-a563ea9d0141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:48.758387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce1b90b-49b3-40d9-9b52-a563ea9d0141 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:48.758675 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce1b90b-49b3-40d9-9b52-a563ea9d0141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:48.758948 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce1b90b-49b3-40d9-9b52-a563ea9d0141 admin admin] PATH_INFO: `/v3/projects/98b04880708e44bfb7e3455e1ebb8e7b/users/2b8fddaa59cc4a90ab3fe0123bdee9d4/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:48.761293 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce1b90b-49b3-40d9-9b52-a563ea9d0141 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98b04880708e44bfb7e3455e1ebb8e7b, user_id=2b8fddaa59cc4a90ab3fe0123bdee9d4, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:48.764152 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce1b90b-49b3-40d9-9b52-a563ea9d0141 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:48.780317 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1362/2763] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:44:48 2022] PUT /identity/v3/projects/98b04880708e44bfb7e3455e1ebb8e7b/users/2b8fddaa59cc4a90ab3fe0123bdee9d4/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:44:48.796940 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f0205-c135-4aac-968b-79547781536a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:48.797698 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f0205-c135-4aac-968b-79547781536a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:48.797698 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f0205-c135-4aac-968b-79547781536a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:48.826986 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a21f0205-c135-4aac-968b-79547781536a None None] MFA Rules not processed for user `2b8fddaa59cc4a90ab3fe0123bdee9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:44:48.828301 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a21f0205-c135-4aac-968b-79547781536a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:48.834333 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:48.906334 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1402/2764] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:44:48 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:44:48.919005 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce1b90b-49b3-40d9-9b52-a563ea9d0141 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:48.921131 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-dce1b90b-49b3-40d9-9b52-a563ea9d0141 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:48.990504 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce1b90b-49b3-40d9-9b52-a563ea9d0141 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:48.995308 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e27f9e-e263-46ba-b6d9-f96a60694860 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:48.995885 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e27f9e-e263-46ba-b6d9-f96a60694860 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:48.996174 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e27f9e-e263-46ba-b6d9-f96a60694860 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:48.996445 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e27f9e-e263-46ba-b6d9-f96a60694860 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:49.001463 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:49.067640 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e27f9e-e263-46ba-b6d9-f96a60694860 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:49.069467 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e27f9e-e263-46ba-b6d9-f96a60694860 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:49.079209 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1363/2765] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:44:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:44:50.646695 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f0205-c135-4aac-968b-79547781536a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:50.648512 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a21f0205-c135-4aac-968b-79547781536a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:50.663150 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f0205-c135-4aac-968b-79547781536a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:50.667577 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c35e58-824a-43f4-890f-724b22dcb584 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:50.668254 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c35e58-824a-43f4-890f-724b22dcb584 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:50.668527 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c35e58-824a-43f4-890f-724b22dcb584 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:50.668803 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c35e58-824a-43f4-890f-724b22dcb584 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:50.745507 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c35e58-824a-43f4-890f-724b22dcb584 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:50.746938 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c35e58-824a-43f4-890f-724b22dcb584 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:50.760332 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1403/2766] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:44:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:44:51.415893 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e27f9e-e263-46ba-b6d9-f96a60694860 service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:51.426631 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e27f9e-e263-46ba-b6d9-f96a60694860 service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:51.430394 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3a0fb0-0882-4454-b43d-19c35c75125a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:51.431052 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a0fb0-0882-4454-b43d-19c35c75125a service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:51.433289 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a0fb0-0882-4454-b43d-19c35c75125a service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:51.433289 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a0fb0-0882-4454-b43d-19c35c75125a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:51.444311 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3a0fb0-0882-4454-b43d-19c35c75125a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:51.445564 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3a0fb0-0882-4454-b43d-19c35c75125a service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:51.456542 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1364/2767] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:44:51 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:44:55.447160 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c35e58-824a-43f4-890f-724b22dcb584 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:55.512176 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c35e58-824a-43f4-890f-724b22dcb584 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:55.519353 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d22312-a7a6-4ee9-91a5-81918e4a4fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:55.520275 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d22312-a7a6-4ee9-91a5-81918e4a4fcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:55.520563 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d22312-a7a6-4ee9-91a5-81918e4a4fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:55.520832 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d22312-a7a6-4ee9-91a5-81918e4a4fcc admin admin] PATH_INFO: `/v3/users/be0d403201ef42eda4ffe8d3ad6539dc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:55.522134 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d22312-a7a6-4ee9-91a5-81918e4a4fcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=be0d403201ef42eda4ffe8d3ad6539dc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:55.524097 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d22312-a7a6-4ee9-91a5-81918e4a4fcc admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:55.581794 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:55.585113 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-98d22312-a7a6-4ee9-91a5-81918e4a4fcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be0d403201ef42eda4ffe8d3ad6539dc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:55.598124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-98d22312-a7a6-4ee9-91a5-81918e4a4fcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be0d403201ef42eda4ffe8d3ad6539dc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:55.598956 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-98d22312-a7a6-4ee9-91a5-81918e4a4fcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be0d403201ef42eda4ffe8d3ad6539dc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:55.599663 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-98d22312-a7a6-4ee9-91a5-81918e4a4fcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be0d403201ef42eda4ffe8d3ad6539dc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:55.607392 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-98d22312-a7a6-4ee9-91a5-81918e4a4fcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be0d403201ef42eda4ffe8d3ad6539dc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:55.623312 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1404/2768] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:44:55 2022] DELETE /identity/v3/users/be0d403201ef42eda4ffe8d3ad6539dc => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:55.628703 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd7722-f5fc-4473-ac3a-0658dbce4f3b None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:55.629225 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd7722-f5fc-4473-ac3a-0658dbce4f3b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:55.629225 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd7722-f5fc-4473-ac3a-0658dbce4f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:55.648167 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-36bd7722-f5fc-4473-ac3a-0658dbce4f3b None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:44:55.649471 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-36bd7722-f5fc-4473-ac3a-0658dbce4f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:55.715977 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1365/2769] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:44:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:44:55.727200 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d22312-a7a6-4ee9-91a5-81918e4a4fcc admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:55.728740 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-98d22312-a7a6-4ee9-91a5-81918e4a4fcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:55.799445 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d22312-a7a6-4ee9-91a5-81918e4a4fcc admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:55.804299 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90973be-efb9-4a93-a0be-d0e74b0933a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:55.805028 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90973be-efb9-4a93-a0be-d0e74b0933a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:55.805424 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90973be-efb9-4a93-a0be-d0e74b0933a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:55.805804 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90973be-efb9-4a93-a0be-d0e74b0933a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:55.818266 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90973be-efb9-4a93-a0be-d0e74b0933a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:55.819738 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90973be-efb9-4a93-a0be-d0e74b0933a6 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:55.829567 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1405/2770] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:44:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:44:56.109560 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bd7722-f5fc-4473-ac3a-0658dbce4f3b None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:56.111145 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-36bd7722-f5fc-4473-ac3a-0658dbce4f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:56.129445 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bd7722-f5fc-4473-ac3a-0658dbce4f3b None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:56.134517 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b671c56f-c702-4398-a248-2f0dbbd7e546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:56.135349 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671c56f-c702-4398-a248-2f0dbbd7e546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:56.135618 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671c56f-c702-4398-a248-2f0dbbd7e546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:56.135919 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671c56f-c702-4398-a248-2f0dbbd7e546 admin admin] PATH_INFO: `/v3/projects/7f22ab4110204152a909fa5526c47307` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:56.137239 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b671c56f-c702-4398-a248-2f0dbbd7e546 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f22ab4110204152a909fa5526c47307)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:56.139214 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b671c56f-c702-4398-a248-2f0dbbd7e546 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:56.190110 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b671c56f-c702-4398-a248-2f0dbbd7e546 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f22ab4110204152a909fa5526c47307', 'request_id': 'req-b671c56f-c702-4398-a248-2f0dbbd7e546'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:56.197608 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b671c56f-c702-4398-a248-2f0dbbd7e546 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f22ab4110204152a909fa5526c47307', 'request_id': 'req-b671c56f-c702-4398-a248-2f0dbbd7e546'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:56.209047 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b671c56f-c702-4398-a248-2f0dbbd7e546 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f22ab4110204152a909fa5526c47307', 'request_id': 'req-b671c56f-c702-4398-a248-2f0dbbd7e546'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:56.220506 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b671c56f-c702-4398-a248-2f0dbbd7e546 admin admin] The token cache is being invalidate because project 7f22ab4110204152a909fa5526c47307 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:44:56.220792 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b671c56f-c702-4398-a248-2f0dbbd7e546 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:56.221480 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b671c56f-c702-4398-a248-2f0dbbd7e546 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:56.223113 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1366/2771] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:44:56 2022] DELETE /identity/v3/projects/7f22ab4110204152a909fa5526c47307 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:56.227226 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90973be-efb9-4a93-a0be-d0e74b0933a6 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:56.237754 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90973be-efb9-4a93-a0be-d0e74b0933a6 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:56.295392 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ec34b-9b84-4d4c-942f-c0d6b4a8ff11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:56.296272 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ec34b-9b84-4d4c-942f-c0d6b4a8ff11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:56.296549 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ec34b-9b84-4d4c-942f-c0d6b4a8ff11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:56.296859 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ec34b-9b84-4d4c-942f-c0d6b4a8ff11 admin admin] PATH_INFO: `/v3/users/f30312d064514b389032c8c035591ea0` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:56.298080 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0ec34b-9b84-4d4c-942f-c0d6b4a8ff11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f30312d064514b389032c8c035591ea0)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:56.300078 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0ec34b-9b84-4d4c-942f-c0d6b4a8ff11 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:56.353446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4b0ec34b-9b84-4d4c-942f-c0d6b4a8ff11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f30312d064514b389032c8c035591ea0'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:56.365446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4b0ec34b-9b84-4d4c-942f-c0d6b4a8ff11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f30312d064514b389032c8c035591ea0'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:56.366233 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4b0ec34b-9b84-4d4c-942f-c0d6b4a8ff11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f30312d064514b389032c8c035591ea0'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:56.366971 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4b0ec34b-9b84-4d4c-942f-c0d6b4a8ff11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f30312d064514b389032c8c035591ea0'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:56.368211 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:56.377458 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4b0ec34b-9b84-4d4c-942f-c0d6b4a8ff11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f30312d064514b389032c8c035591ea0'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:56.392583 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1406/2772] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:44:56 2022] DELETE /identity/v3/users/f30312d064514b389032c8c035591ea0 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:56.691061 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b671c56f-c702-4398-a248-2f0dbbd7e546 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:56.693019 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b671c56f-c702-4398-a248-2f0dbbd7e546 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:56.708134 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b671c56f-c702-4398-a248-2f0dbbd7e546 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:56.722432 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:56.772446 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0924f6-c96f-4961-9134-ab2fec739141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:56.773335 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0924f6-c96f-4961-9134-ab2fec739141 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:56.773617 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0924f6-c96f-4961-9134-ab2fec739141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:56.773913 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0924f6-c96f-4961-9134-ab2fec739141 admin admin] PATH_INFO: `/v3/projects/303be763807c4e27911870f876fa96eb` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:56.775152 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0924f6-c96f-4961-9134-ab2fec739141 admin admin] RBAC: Authorizing `identity:delete_project(project_id=303be763807c4e27911870f876fa96eb)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:56.777108 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0924f6-c96f-4961-9134-ab2fec739141 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:56.828462 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5b0924f6-c96f-4961-9134-ab2fec739141 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '303be763807c4e27911870f876fa96eb', 'request_id': 'req-5b0924f6-c96f-4961-9134-ab2fec739141'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:56.834852 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5b0924f6-c96f-4961-9134-ab2fec739141 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '303be763807c4e27911870f876fa96eb', 'request_id': 'req-5b0924f6-c96f-4961-9134-ab2fec739141'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:56.844960 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5b0924f6-c96f-4961-9134-ab2fec739141 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '303be763807c4e27911870f876fa96eb', 'request_id': 'req-5b0924f6-c96f-4961-9134-ab2fec739141'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:56.859726 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5b0924f6-c96f-4961-9134-ab2fec739141 admin admin] The token cache is being invalidate because project 303be763807c4e27911870f876fa96eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:44:56.860011 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5b0924f6-c96f-4961-9134-ab2fec739141 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:56.860806 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5b0924f6-c96f-4961-9134-ab2fec739141 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:56.862526 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1367/2773] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:44:56 2022] DELETE /identity/v3/projects/303be763807c4e27911870f876fa96eb => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:56.866897 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ec34b-9b84-4d4c-942f-c0d6b4a8ff11 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:56.868413 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4b0ec34b-9b84-4d4c-942f-c0d6b4a8ff11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:56.882639 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ec34b-9b84-4d4c-942f-c0d6b4a8ff11 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:56.950396 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f55ad-3cfe-4249-8626-3423f3bf313a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:56.951312 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f55ad-3cfe-4249-8626-3423f3bf313a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:56.951598 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f55ad-3cfe-4249-8626-3423f3bf313a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:56.951897 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f55ad-3cfe-4249-8626-3423f3bf313a admin admin] PATH_INFO: `/v3/users/2b8fddaa59cc4a90ab3fe0123bdee9d4` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:56.953370 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333f55ad-3cfe-4249-8626-3423f3bf313a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b8fddaa59cc4a90ab3fe0123bdee9d4)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:56.955397 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333f55ad-3cfe-4249-8626-3423f3bf313a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:57.020007 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-333f55ad-3cfe-4249-8626-3423f3bf313a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b8fddaa59cc4a90ab3fe0123bdee9d4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:57.033089 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-333f55ad-3cfe-4249-8626-3423f3bf313a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b8fddaa59cc4a90ab3fe0123bdee9d4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:57.033946 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-333f55ad-3cfe-4249-8626-3423f3bf313a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b8fddaa59cc4a90ab3fe0123bdee9d4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:57.034707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-333f55ad-3cfe-4249-8626-3423f3bf313a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b8fddaa59cc4a90ab3fe0123bdee9d4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:57.042950 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-333f55ad-3cfe-4249-8626-3423f3bf313a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b8fddaa59cc4a90ab3fe0123bdee9d4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:57.057521 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1407/2774] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:44:56 2022] DELETE /identity/v3/users/2b8fddaa59cc4a90ab3fe0123bdee9d4 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:44:57.376724 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0924f6-c96f-4961-9134-ab2fec739141 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:44:57.379108 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5b0924f6-c96f-4961-9134-ab2fec739141 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:44:57.405615 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0924f6-c96f-4961-9134-ab2fec739141 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:44:57.472464 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e11107f-0407-4654-b416-f8d12ae590f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:44:57.473288 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11107f-0407-4654-b416-f8d12ae590f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:44:57.473540 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11107f-0407-4654-b416-f8d12ae590f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:44:57.473810 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11107f-0407-4654-b416-f8d12ae590f7 admin admin] PATH_INFO: `/v3/projects/98b04880708e44bfb7e3455e1ebb8e7b` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:44:57.478081 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e11107f-0407-4654-b416-f8d12ae590f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98b04880708e44bfb7e3455e1ebb8e7b)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:44:57.480265 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e11107f-0407-4654-b416-f8d12ae590f7 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:44:57.528014 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:44:57.529371 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7e11107f-0407-4654-b416-f8d12ae590f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98b04880708e44bfb7e3455e1ebb8e7b', 'request_id': 'req-7e11107f-0407-4654-b416-f8d12ae590f7'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:57.539991 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7e11107f-0407-4654-b416-f8d12ae590f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98b04880708e44bfb7e3455e1ebb8e7b', 'request_id': 'req-7e11107f-0407-4654-b416-f8d12ae590f7'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:57.553181 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7e11107f-0407-4654-b416-f8d12ae590f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98b04880708e44bfb7e3455e1ebb8e7b', 'request_id': 'req-7e11107f-0407-4654-b416-f8d12ae590f7'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:57.564151 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7e11107f-0407-4654-b416-f8d12ae590f7 admin admin] The token cache is being invalidate because project 98b04880708e44bfb7e3455e1ebb8e7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:44:57.564518 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7e11107f-0407-4654-b416-f8d12ae590f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:57.565345 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-7e11107f-0407-4654-b416-f8d12ae590f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:44:57.567133 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1368/2775] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:44:57 2022] DELETE /identity/v3/projects/98b04880708e44bfb7e3455e1ebb8e7b => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:45:07.053776 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=59,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:45:07.561100 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=7,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:45:20.592035 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f55ad-3cfe-4249-8626-3423f3bf313a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:45:20.594629 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-333f55ad-3cfe-4249-8626-3423f3bf313a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:45:20.660800 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f55ad-3cfe-4249-8626-3423f3bf313a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:45:20.714808 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72685739-65d9-4fd3-bd75-5c95720cf114 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:45:20.715484 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72685739-65d9-4fd3-bd75-5c95720cf114 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:45:20.715770 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72685739-65d9-4fd3-bd75-5c95720cf114 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:45:20.716047 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72685739-65d9-4fd3-bd75-5c95720cf114 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:45:20.717979 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-72685739-65d9-4fd3-bd75-5c95720cf114 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:45:20.804279 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72685739-65d9-4fd3-bd75-5c95720cf114 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:45:20.805739 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72685739-65d9-4fd3-bd75-5c95720cf114 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:45:20.826274 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1408/2776] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:45:20 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:45:30.413924 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e11107f-0407-4654-b416-f8d12ae590f7 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:45:30.415777 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7e11107f-0407-4654-b416-f8d12ae590f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:45:30.504549 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e11107f-0407-4654-b416-f8d12ae590f7 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:45:30.511759 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eaf5bc-4d6f-42da-aa9f-cf00dd906993 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:45:30.512423 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eaf5bc-4d6f-42da-aa9f-cf00dd906993 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:45:30.512673 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eaf5bc-4d6f-42da-aa9f-cf00dd906993 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:45:30.512942 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eaf5bc-4d6f-42da-aa9f-cf00dd906993 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:45:30.514913 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e6eaf5bc-4d6f-42da-aa9f-cf00dd906993 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:45:30.527530 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eaf5bc-4d6f-42da-aa9f-cf00dd906993 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:45:30.529126 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eaf5bc-4d6f-42da-aa9f-cf00dd906993 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:45:30.540645 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1369/2777] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:45:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:45:30.824471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=81 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:45:33.459617 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72685739-65d9-4fd3-bd75-5c95720cf114 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:45:33.462816 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-72685739-65d9-4fd3-bd75-5c95720cf114 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:45:33.476888 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72685739-65d9-4fd3-bd75-5c95720cf114 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:45:33.481529 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c40b3-fdd2-4889-bfc8-583f07c360d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:45:33.482216 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c40b3-fdd2-4889-bfc8-583f07c360d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:45:33.482533 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c40b3-fdd2-4889-bfc8-583f07c360d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:45:33.482834 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c40b3-fdd2-4889-bfc8-583f07c360d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:45:33.485019 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d56c40b3-fdd2-4889-bfc8-583f07c360d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:45:33.554297 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56c40b3-fdd2-4889-bfc8-583f07c360d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:45:33.555810 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56c40b3-fdd2-4889-bfc8-583f07c360d9 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:45:33.567780 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1409/2778] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:45:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:45:40.394590 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eaf5bc-4d6f-42da-aa9f-cf00dd906993 service nova] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:45:40.396152 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e6eaf5bc-4d6f-42da-aa9f-cf00dd906993 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:45:40.411222 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eaf5bc-4d6f-42da-aa9f-cf00dd906993 service nova] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:45:40.415871 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea023d5b-a498-4727-a799-4832081dbf76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:45:40.416755 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea023d5b-a498-4727-a799-4832081dbf76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:45:40.417037 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea023d5b-a498-4727-a799-4832081dbf76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:45:40.417308 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea023d5b-a498-4727-a799-4832081dbf76 admin admin] PATH_INFO: `/v3/users/0a58f82117ab437cab5607c204c86547` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:45:40.418566 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea023d5b-a498-4727-a799-4832081dbf76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a58f82117ab437cab5607c204c86547)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:45:40.420524 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea023d5b-a498-4727-a799-4832081dbf76 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:45:40.472043 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ea023d5b-a498-4727-a799-4832081dbf76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a58f82117ab437cab5607c204c86547'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:45:40.472603 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ea023d5b-a498-4727-a799-4832081dbf76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a58f82117ab437cab5607c204c86547'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:45:40.485936 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ea023d5b-a498-4727-a799-4832081dbf76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a58f82117ab437cab5607c204c86547'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:45:40.493295 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ea023d5b-a498-4727-a799-4832081dbf76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a58f82117ab437cab5607c204c86547'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:45:40.502355 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ea023d5b-a498-4727-a799-4832081dbf76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a58f82117ab437cab5607c204c86547'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:45:40.509500 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1370/2779] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:45:40 2022] DELETE /identity/v3/users/0a58f82117ab437cab5607c204c86547 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 21:45:40.692413 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c40b3-fdd2-4889-bfc8-583f07c360d9 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:45:40.704650 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c40b3-fdd2-4889-bfc8-583f07c360d9 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:45:40.766166 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f03626-2897-4541-9140-83df9eebacbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:45:40.767209 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f03626-2897-4541-9140-83df9eebacbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:45:40.767490 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f03626-2897-4541-9140-83df9eebacbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:45:40.767784 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f03626-2897-4541-9140-83df9eebacbb admin admin] PATH_INFO: `/v3/projects/4951b93f86e1485692a4aae7b9ece8f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:45:40.769081 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f03626-2897-4541-9140-83df9eebacbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=4951b93f86e1485692a4aae7b9ece8f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:45:40.771062 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f03626-2897-4541-9140-83df9eebacbb admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:45:40.830611 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-30f03626-2897-4541-9140-83df9eebacbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4951b93f86e1485692a4aae7b9ece8f6', 'request_id': 'req-30f03626-2897-4541-9140-83df9eebacbb'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:45:40.836938 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-30f03626-2897-4541-9140-83df9eebacbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4951b93f86e1485692a4aae7b9ece8f6', 'request_id': 'req-30f03626-2897-4541-9140-83df9eebacbb'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:45:40.842663 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-30f03626-2897-4541-9140-83df9eebacbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4951b93f86e1485692a4aae7b9ece8f6', 'request_id': 'req-30f03626-2897-4541-9140-83df9eebacbb'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:45:40.862878 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-30f03626-2897-4541-9140-83df9eebacbb admin admin] The token cache is being invalidate because project 4951b93f86e1485692a4aae7b9ece8f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:45:40.863187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-30f03626-2897-4541-9140-83df9eebacbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:45:40.863905 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-30f03626-2897-4541-9140-83df9eebacbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:45:40.865604 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1410/2780] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:45:40 2022] DELETE /identity/v3/projects/4951b93f86e1485692a4aae7b9ece8f6 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:45:40.869763 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea023d5b-a498-4727-a799-4832081dbf76 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:45:40.871313 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ea023d5b-a498-4727-a799-4832081dbf76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:45:40.885782 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea023d5b-a498-4727-a799-4832081dbf76 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:45:40.937656 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:45:40.952447 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77972a76-9da2-47ae-b218-092ee6088864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:45:40.953350 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77972a76-9da2-47ae-b218-092ee6088864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:45:40.953642 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77972a76-9da2-47ae-b218-092ee6088864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:45:40.954167 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77972a76-9da2-47ae-b218-092ee6088864 admin admin] PATH_INFO: `/v3/users/a991019b81a843d38987f889fec0eec9` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:45:40.969475 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77972a76-9da2-47ae-b218-092ee6088864 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a991019b81a843d38987f889fec0eec9)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:45:40.971502 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77972a76-9da2-47ae-b218-092ee6088864 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:45:41.028090 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-77972a76-9da2-47ae-b218-092ee6088864 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a991019b81a843d38987f889fec0eec9'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:45:41.028972 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-77972a76-9da2-47ae-b218-092ee6088864 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a991019b81a843d38987f889fec0eec9'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:45:41.043824 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-77972a76-9da2-47ae-b218-092ee6088864 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a991019b81a843d38987f889fec0eec9'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:45:41.054181 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-77972a76-9da2-47ae-b218-092ee6088864 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a991019b81a843d38987f889fec0eec9'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:45:41.065034 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-77972a76-9da2-47ae-b218-092ee6088864 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a991019b81a843d38987f889fec0eec9'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:45:41.072427 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1371/2781] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:45:40 2022] DELETE /identity/v3/users/a991019b81a843d38987f889fec0eec9 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:45:41.261305 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f03626-2897-4541-9140-83df9eebacbb admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:45:41.266595 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-30f03626-2897-4541-9140-83df9eebacbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:45:41.288458 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f03626-2897-4541-9140-83df9eebacbb admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:45:41.318974 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:45:41.366047 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d9a29e-c0a4-4353-a22c-82b769c28adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:45:41.366962 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9a29e-c0a4-4353-a22c-82b769c28adc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:45:41.367243 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9a29e-c0a4-4353-a22c-82b769c28adc admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:45:41.367517 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9a29e-c0a4-4353-a22c-82b769c28adc admin admin] PATH_INFO: `/v3/projects/8b2c1a02d52a4033956d358959ba1e6f` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:45:41.379191 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d9a29e-c0a4-4353-a22c-82b769c28adc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b2c1a02d52a4033956d358959ba1e6f)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:45:41.381124 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d9a29e-c0a4-4353-a22c-82b769c28adc admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:45:41.432345 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-79d9a29e-c0a4-4353-a22c-82b769c28adc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b2c1a02d52a4033956d358959ba1e6f', 'request_id': 'req-79d9a29e-c0a4-4353-a22c-82b769c28adc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:45:41.439435 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-79d9a29e-c0a4-4353-a22c-82b769c28adc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b2c1a02d52a4033956d358959ba1e6f', 'request_id': 'req-79d9a29e-c0a4-4353-a22c-82b769c28adc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:45:41.445224 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-79d9a29e-c0a4-4353-a22c-82b769c28adc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b2c1a02d52a4033956d358959ba1e6f', 'request_id': 'req-79d9a29e-c0a4-4353-a22c-82b769c28adc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:45:41.459548 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-79d9a29e-c0a4-4353-a22c-82b769c28adc admin admin] The token cache is being invalidate because project 8b2c1a02d52a4033956d358959ba1e6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:45:41.459840 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-79d9a29e-c0a4-4353-a22c-82b769c28adc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:45:41.460545 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-79d9a29e-c0a4-4353-a22c-82b769c28adc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:45:41.462181 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1411/2782] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:45:41 2022] DELETE /identity/v3/projects/8b2c1a02d52a4033956d358959ba1e6f => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:45:51.068294 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=35,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:45:51.456600 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=44,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:46:02.244630 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6e6be-3836-47e6-97bf-0c52244393f9 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:02.245356 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6e6be-3836-47e6-97bf-0c52244393f9 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:02.245356 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6e6be-3836-47e6-97bf-0c52244393f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:02.263751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-e2b6e6be-3836-47e6-97bf-0c52244393f9 None None] MFA Rules not processed for user `b9bd7a1a0e804d23aaa68153549672c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:46:02.264959 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e2b6e6be-3836-47e6-97bf-0c52244393f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:46:02.326787 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1372/2783] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:46:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:46:02.664409 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d9a29e-c0a4-4353-a22c-82b769c28adc admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:46:02.665935 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-79d9a29e-c0a4-4353-a22c-82b769c28adc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:46:02.730257 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d9a29e-c0a4-4353-a22c-82b769c28adc admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:46:02.734812 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fdcc95-5acc-4aac-ab0e-cd1fe9de9109 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:46:02.735461 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fdcc95-5acc-4aac-ab0e-cd1fe9de9109 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:02.735749 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fdcc95-5acc-4aac-ab0e-cd1fe9de9109 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:02.736006 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fdcc95-5acc-4aac-ab0e-cd1fe9de9109 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:02.738169 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-77fdcc95-5acc-4aac-ab0e-cd1fe9de9109 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:46:02.748297 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fdcc95-5acc-4aac-ab0e-cd1fe9de9109 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:46:02.749722 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fdcc95-5acc-4aac-ab0e-cd1fe9de9109 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:46:02.757916 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1412/2784] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:46:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:46:03.592968 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b6e6be-3836-47e6-97bf-0c52244393f9 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:46:03.594427 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e2b6e6be-3836-47e6-97bf-0c52244393f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:46:03.692861 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b6e6be-3836-47e6-97bf-0c52244393f9 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:46:03.698704 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b856d6cf-e567-43f3-8893-5d26910537bc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6143bddc05a46ea972709576c7ad95f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:46:03.699358 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b856d6cf-e567-43f3-8893-5d26910537bc service placement] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:46:03.699652 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b856d6cf-e567-43f3-8893-5d26910537bc service placement] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:46:03.699929 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b856d6cf-e567-43f3-8893-5d26910537bc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:46:03.701887 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b856d6cf-e567-43f3-8893-5d26910537bc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:46:03.715591 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b856d6cf-e567-43f3-8893-5d26910537bc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:46:03.717398 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b856d6cf-e567-43f3-8893-5d26910537bc service placement] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:46:03.729073 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1373/2785] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:46:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:46:12.754634 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=26 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:46:13.723941 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=66 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:30.156094 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fdcc95-5acc-4aac-ab0e-cd1fe9de9109 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:30.157591 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-77fdcc95-5acc-4aac-ab0e-cd1fe9de9109 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:30.217372 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fdcc95-5acc-4aac-ab0e-cd1fe9de9109 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:30.221468 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e846e-bf23-454d-a722-231f08d0c142 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:30.222118 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e846e-bf23-454d-a722-231f08d0c142 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:30.222419 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e846e-bf23-454d-a722-231f08d0c142 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:30.222699 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e846e-bf23-454d-a722-231f08d0c142 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:30.224875 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-029e846e-bf23-454d-a722-231f08d0c142 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:30.329300 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029e846e-bf23-454d-a722-231f08d0c142 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:30.331367 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029e846e-bf23-454d-a722-231f08d0c142 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:30.353657 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1413/2786] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:47:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:47:31.791457 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b856d6cf-e567-43f3-8893-5d26910537bc service placement] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:31.794482 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b856d6cf-e567-43f3-8893-5d26910537bc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:31.810980 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b856d6cf-e567-43f3-8893-5d26910537bc service placement] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:31.815454 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9a42dc-10f0-4ad3-86c5-fc2226c2302a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:31.816096 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a42dc-10f0-4ad3-86c5-fc2226c2302a service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:31.816385 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a42dc-10f0-4ad3-86c5-fc2226c2302a service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:31.816665 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a42dc-10f0-4ad3-86c5-fc2226c2302a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:31.818571 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6d9a42dc-10f0-4ad3-86c5-fc2226c2302a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:31.834772 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9a42dc-10f0-4ad3-86c5-fc2226c2302a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:31.836258 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9a42dc-10f0-4ad3-86c5-fc2226c2302a service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:31.849620 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1374/2787] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:47:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:47:40.350983 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=63 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:41.843256 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=6 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:41.936167 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cebc7f-f16d-4cf4-aacb-8b7a2170ad5f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:41.937087 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cebc7f-f16d-4cf4-aacb-8b7a2170ad5f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:41.937087 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cebc7f-f16d-4cf4-aacb-8b7a2170ad5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:41.954506 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e9cebc7f-f16d-4cf4-aacb-8b7a2170ad5f None None] MFA Rules not processed for user `b9bd7a1a0e804d23aaa68153549672c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:47:41.955708 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e9cebc7f-f16d-4cf4-aacb-8b7a2170ad5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:41.963027 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1414/2788] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:47:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:47:42.304563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9a42dc-10f0-4ad3-86c5-fc2226c2302a service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:42.314416 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9a42dc-10f0-4ad3-86c5-fc2226c2302a service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:42.318508 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d655d1d-730e-43ea-a86e-eec71d4f56fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:42.319157 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d655d1d-730e-43ea-a86e-eec71d4f56fa service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:42.319441 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d655d1d-730e-43ea-a86e-eec71d4f56fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:42.319708 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d655d1d-730e-43ea-a86e-eec71d4f56fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:42.321885 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1d655d1d-730e-43ea-a86e-eec71d4f56fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:42.333227 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d655d1d-730e-43ea-a86e-eec71d4f56fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:42.334631 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d655d1d-730e-43ea-a86e-eec71d4f56fa service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:42.343435 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1375/2789] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:47:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:47:44.367560 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cebc7f-f16d-4cf4-aacb-8b7a2170ad5f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:47:44.379170 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cebc7f-f16d-4cf4-aacb-8b7a2170ad5f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:47:44.383002 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af545a-853b-46fe-88d3-c482993c8bee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:47:44.383682 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af545a-853b-46fe-88d3-c482993c8bee service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:47:44.383969 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af545a-853b-46fe-88d3-c482993c8bee service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:47:44.384245 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af545a-853b-46fe-88d3-c482993c8bee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:47:44.386219 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d8af545a-853b-46fe-88d3-c482993c8bee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:47:44.397330 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8af545a-853b-46fe-88d3-c482993c8bee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:47:44.398863 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8af545a-853b-46fe-88d3-c482993c8bee service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:47:44.409339 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1415/2790] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:47:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:47:52.339865 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=6 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:47:54.405174 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=9 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:19.926515 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d655d1d-730e-43ea-a86e-eec71d4f56fa service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:19.941716 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d655d1d-730e-43ea-a86e-eec71d4f56fa service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:19.946787 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64382478-4bb3-47d6-8142-90664f7c2f64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:19.947477 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64382478-4bb3-47d6-8142-90664f7c2f64 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:19.947765 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64382478-4bb3-47d6-8142-90664f7c2f64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:19.948048 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64382478-4bb3-47d6-8142-90664f7c2f64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:19.950226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-64382478-4bb3-47d6-8142-90664f7c2f64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:49:19.962531 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64382478-4bb3-47d6-8142-90664f7c2f64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:19.963982 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64382478-4bb3-47d6-8142-90664f7c2f64 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:19.973559 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1376/2791] 10.222.0.65 () {60 vars in 1396 bytes} [Thu May 19 21:49:19 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3252 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:49:25.468076 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f71ce8-6c03-4926-ac59-38ad6eb85d63 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:25.469036 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f71ce8-6c03-4926-ac59-38ad6eb85d63 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:25.469596 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f71ce8-6c03-4926-ac59-38ad6eb85d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:25.493261 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d2f71ce8-6c03-4926-ac59-38ad6eb85d63 None None] MFA Rules not processed for user `b9bd7a1a0e804d23aaa68153549672c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:49:25.494990 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d2f71ce8-6c03-4926-ac59-38ad6eb85d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:49:25.505641 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1416/2792] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:49:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:49:26.161793 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64382478-4bb3-47d6-8142-90664f7c2f64 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:26.175861 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64382478-4bb3-47d6-8142-90664f7c2f64 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:26.181337 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23246498-92a6-4581-8573-e3e0d3f7548c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6143bddc05a46ea972709576c7ad95f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:26.182046 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23246498-92a6-4581-8573-e3e0d3f7548c service placement] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:26.182370 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23246498-92a6-4581-8573-e3e0d3f7548c service placement] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:26.182634 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23246498-92a6-4581-8573-e3e0d3f7548c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:26.184461 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-23246498-92a6-4581-8573-e3e0d3f7548c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:49:26.199050 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23246498-92a6-4581-8573-e3e0d3f7548c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:26.200704 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23246498-92a6-4581-8573-e3e0d3f7548c service placement] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:26.211959 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1377/2793] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:49:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:49:34.315664 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f71ce8-6c03-4926-ac59-38ad6eb85d63 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:49:34.317531 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d2f71ce8-6c03-4926-ac59-38ad6eb85d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:49:34.334392 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f71ce8-6c03-4926-ac59-38ad6eb85d63 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:49:34.340512 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b5f5b-8db8-4067-9418-dd0c1ecc9fa9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:49:34.341187 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b5f5b-8db8-4067-9418-dd0c1ecc9fa9 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:49:34.341467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b5f5b-8db8-4067-9418-dd0c1ecc9fa9 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:49:34.341726 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b5f5b-8db8-4067-9418-dd0c1ecc9fa9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:49:34.343596 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-531b5f5b-8db8-4067-9418-dd0c1ecc9fa9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:49:34.357461 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531b5f5b-8db8-4067-9418-dd0c1ecc9fa9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:49:34.357975 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531b5f5b-8db8-4067-9418-dd0c1ecc9fa9 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:49:34.369692 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1417/2794] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:49:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:49:36.207534 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=12 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:49:44.364516 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=9 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:06.714869 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23246498-92a6-4581-8573-e3e0d3f7548c service placement] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:06.717347 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-23246498-92a6-4581-8573-e3e0d3f7548c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:51:06.740554 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23246498-92a6-4581-8573-e3e0d3f7548c service placement] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:06.753803 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaa610d-4b83-4054-bfcb-b7474ce53689 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:06.754776 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa610d-4b83-4054-bfcb-b7474ce53689 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:06.755060 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa610d-4b83-4054-bfcb-b7474ce53689 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:06.755339 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa610d-4b83-4054-bfcb-b7474ce53689 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:06.757244 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-aeaa610d-4b83-4054-bfcb-b7474ce53689 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:51:06.771613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaa610d-4b83-4054-bfcb-b7474ce53689 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:06.773529 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaa610d-4b83-4054-bfcb-b7474ce53689 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:06.797505 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1378/2795] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:51:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:51:06.879416 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b5f5b-8db8-4067-9418-dd0c1ecc9fa9 service nova] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:06.881080 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-531b5f5b-8db8-4067-9418-dd0c1ecc9fa9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:51:06.896585 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b5f5b-8db8-4067-9418-dd0c1ecc9fa9 service nova] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:06.901966 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b64a2-210f-4f87-be93-b588f960e8d9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6143bddc05a46ea972709576c7ad95f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:06.902915 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b64a2-210f-4f87-be93-b588f960e8d9 service placement] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:06.903299 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b64a2-210f-4f87-be93-b588f960e8d9 service placement] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:06.903664 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b64a2-210f-4f87-be93-b588f960e8d9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:06.914379 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840b64a2-210f-4f87-be93-b588f960e8d9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:06.916178 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840b64a2-210f-4f87-be93-b588f960e8d9 service placement] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:06.927691 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1418/2796] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:51:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:51:12.215043 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaa610d-4b83-4054-bfcb-b7474ce53689 service nova] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:12.225156 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaa610d-4b83-4054-bfcb-b7474ce53689 service nova] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:12.229063 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed6e99-f6a8-43bd-94c7-3d2b346390fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:12.229701 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed6e99-f6a8-43bd-94c7-3d2b346390fe service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:12.229984 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed6e99-f6a8-43bd-94c7-3d2b346390fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:12.230283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed6e99-f6a8-43bd-94c7-3d2b346390fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:12.232194 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-22ed6e99-f6a8-43bd-94c7-3d2b346390fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:51:12.301379 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ed6e99-f6a8-43bd-94c7-3d2b346390fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:12.303339 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ed6e99-f6a8-43bd-94c7-3d2b346390fe service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:12.316393 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1379/2797] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:51:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:51:15.330104 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b64a2-210f-4f87-be93-b588f960e8d9 service placement] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:15.341274 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b64a2-210f-4f87-be93-b588f960e8d9 service placement] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:15.346615 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ce058f-a1b0-4454-9b30-68eda185de6d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6143bddc05a46ea972709576c7ad95f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:15.347647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ce058f-a1b0-4454-9b30-68eda185de6d service placement] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:15.348231 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ce058f-a1b0-4454-9b30-68eda185de6d service placement] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:15.348740 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ce058f-a1b0-4454-9b30-68eda185de6d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:15.351180 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-44ce058f-a1b0-4454-9b30-68eda185de6d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:51:15.365267 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ce058f-a1b0-4454-9b30-68eda185de6d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:15.367260 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ce058f-a1b0-4454-9b30-68eda185de6d service placement] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:15.386365 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1419/2798] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:51:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:51:15.418528 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed6e99-f6a8-43bd-94c7-3d2b346390fe service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:15.420474 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-22ed6e99-f6a8-43bd-94c7-3d2b346390fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:51:15.435620 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed6e99-f6a8-43bd-94c7-3d2b346390fe service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:15.446461 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e7a518-3658-431c-81b0-b142be5bc334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:15.447551 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e7a518-3658-431c-81b0-b142be5bc334 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:15.448024 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e7a518-3658-431c-81b0-b142be5bc334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:15.448302 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e7a518-3658-431c-81b0-b142be5bc334 admin admin] PATH_INFO: `/v3/users/b9bd7a1a0e804d23aaa68153549672c9` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:15.449526 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e7a518-3658-431c-81b0-b142be5bc334 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9bd7a1a0e804d23aaa68153549672c9)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:15.451501 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e7a518-3658-431c-81b0-b142be5bc334 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:15.505797 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-23e7a518-3658-431c-81b0-b142be5bc334 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9bd7a1a0e804d23aaa68153549672c9'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:51:15.506495 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-23e7a518-3658-431c-81b0-b142be5bc334 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9bd7a1a0e804d23aaa68153549672c9'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:51:15.518361 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-23e7a518-3658-431c-81b0-b142be5bc334 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9bd7a1a0e804d23aaa68153549672c9'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:51:15.527043 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-23e7a518-3658-431c-81b0-b142be5bc334 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9bd7a1a0e804d23aaa68153549672c9'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:51:15.534962 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-23e7a518-3658-431c-81b0-b142be5bc334 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9bd7a1a0e804d23aaa68153549672c9'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:51:15.541342 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1380/2799] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:51:15 2022] DELETE /identity/v3/users/b9bd7a1a0e804d23aaa68153549672c9 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:51:15.740762 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ce058f-a1b0-4454-9b30-68eda185de6d service placement] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:15.752054 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ce058f-a1b0-4454-9b30-68eda185de6d service placement] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:15.813483 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab4f34-5c21-4a82-89dd-f591e5ef9de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:15.814399 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab4f34-5c21-4a82-89dd-f591e5ef9de6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:15.814669 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab4f34-5c21-4a82-89dd-f591e5ef9de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:15.814944 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab4f34-5c21-4a82-89dd-f591e5ef9de6 admin admin] PATH_INFO: `/v3/projects/9a4ad8679d5446b591d4083529f992cc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:15.816160 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab4f34-5c21-4a82-89dd-f591e5ef9de6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a4ad8679d5446b591d4083529f992cc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:15.819153 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab4f34-5c21-4a82-89dd-f591e5ef9de6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:15.869797 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b9ab4f34-5c21-4a82-89dd-f591e5ef9de6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a4ad8679d5446b591d4083529f992cc', 'request_id': 'req-b9ab4f34-5c21-4a82-89dd-f591e5ef9de6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:51:15.878607 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b9ab4f34-5c21-4a82-89dd-f591e5ef9de6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a4ad8679d5446b591d4083529f992cc', 'request_id': 'req-b9ab4f34-5c21-4a82-89dd-f591e5ef9de6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:51:15.885328 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b9ab4f34-5c21-4a82-89dd-f591e5ef9de6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a4ad8679d5446b591d4083529f992cc', 'request_id': 'req-b9ab4f34-5c21-4a82-89dd-f591e5ef9de6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:51:15.899839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b9ab4f34-5c21-4a82-89dd-f591e5ef9de6 admin admin] The token cache is being invalidate because project 9a4ad8679d5446b591d4083529f992cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:51:15.900154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b9ab4f34-5c21-4a82-89dd-f591e5ef9de6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:51:15.900810 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b9ab4f34-5c21-4a82-89dd-f591e5ef9de6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:51:15.902461 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1420/2800] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:51:15 2022] DELETE /identity/v3/projects/9a4ad8679d5446b591d4083529f992cc => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:51:15.907507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5d942-72fc-42fd-a825-3c3fa3e0f818 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:15.907797 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5d942-72fc-42fd-a825-3c3fa3e0f818 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:15.908074 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5d942-72fc-42fd-a825-3c3fa3e0f818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:15.926225 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-76b5d942-72fc-42fd-a825-3c3fa3e0f818 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:51:15.927510 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-76b5d942-72fc-42fd-a825-3c3fa3e0f818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:51:15.979438 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:15.993386 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1381/2801] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:51:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:51:16.001566 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87f194-48e2-41d3-8aec-c4970a054f71 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:16.001864 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87f194-48e2-41d3-8aec-c4970a054f71 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:16.002148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87f194-48e2-41d3-8aec-c4970a054f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:16.022766 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-4d87f194-48e2-41d3-8aec-c4970a054f71 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:51:16.024509 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4d87f194-48e2-41d3-8aec-c4970a054f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:51:16.034219 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1421/2802] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:51:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:51:16.038842 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b5d942-72fc-42fd-a825-3c3fa3e0f818 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:16.049024 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b5d942-72fc-42fd-a825-3c3fa3e0f818 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:16.055463 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b129e9a0-6b41-4563-9901-3da91ef2a5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:16.056138 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129e9a0-6b41-4563-9901-3da91ef2a5dd admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:16.056431 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129e9a0-6b41-4563-9901-3da91ef2a5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:16.056702 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129e9a0-6b41-4563-9901-3da91ef2a5dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:16.057445 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b129e9a0-6b41-4563-9901-3da91ef2a5dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:16.057752 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b129e9a0-6b41-4563-9901-3da91ef2a5dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:16.059418 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b129e9a0-6b41-4563-9901-3da91ef2a5dd admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:16.072321 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1382/2803] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:51:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:51:16.076902 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87f194-48e2-41d3-8aec-c4970a054f71 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:16.087226 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87f194-48e2-41d3-8aec-c4970a054f71 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:16.092854 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02818e1c-e087-4e57-bc78-8a2d03d8f126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:16.093515 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02818e1c-e087-4e57-bc78-8a2d03d8f126 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:16.093802 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02818e1c-e087-4e57-bc78-8a2d03d8f126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:16.094073 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02818e1c-e087-4e57-bc78-8a2d03d8f126 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:16.094798 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02818e1c-e087-4e57-bc78-8a2d03d8f126 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:16.096707 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02818e1c-e087-4e57-bc78-8a2d03d8f126 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:16.135622 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1422/2804] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:51:16 2022] POST /identity/v3/projects => generated 367 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:51:16.137051 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b129e9a0-6b41-4563-9901-3da91ef2a5dd admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:16.147437 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b129e9a0-6b41-4563-9901-3da91ef2a5dd admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:16.221215 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df38c132-1214-478d-8927-40a6d942aac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:16.221912 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38c132-1214-478d-8927-40a6d942aac7 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:16.222196 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38c132-1214-478d-8927-40a6d942aac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:16.222496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38c132-1214-478d-8927-40a6d942aac7 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:16.223206 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df38c132-1214-478d-8927-40a6d942aac7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:16.225886 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df38c132-1214-478d-8927-40a6d942aac7 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:16.249184 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1383/2805] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:51:16 2022] POST /identity/v3/users => generated 284 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:51:16.253451 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02818e1c-e087-4e57-bc78-8a2d03d8f126 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:16.264090 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02818e1c-e087-4e57-bc78-8a2d03d8f126 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:16.269386 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71665037-73b9-41f1-a5b2-592a945d43ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:16.270055 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71665037-73b9-41f1-a5b2-592a945d43ad admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:16.270381 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71665037-73b9-41f1-a5b2-592a945d43ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:16.270775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71665037-73b9-41f1-a5b2-592a945d43ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:16.271501 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71665037-73b9-41f1-a5b2-592a945d43ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:16.273101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71665037-73b9-41f1-a5b2-592a945d43ad admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:16.283915 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1423/2806] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:51:16.288781 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df38c132-1214-478d-8927-40a6d942aac7 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:16.303160 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df38c132-1214-478d-8927-40a6d942aac7 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:16.309096 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8192e680-8ead-4976-abfd-60ed36ea310e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:16.309769 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8192e680-8ead-4976-abfd-60ed36ea310e admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:16.310056 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8192e680-8ead-4976-abfd-60ed36ea310e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:16.310385 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8192e680-8ead-4976-abfd-60ed36ea310e admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:16.311120 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8192e680-8ead-4976-abfd-60ed36ea310e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:16.312743 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8192e680-8ead-4976-abfd-60ed36ea310e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:16.323659 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1384/2807] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 21:51:16.330032 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71665037-73b9-41f1-a5b2-592a945d43ad admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:16.343822 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71665037-73b9-41f1-a5b2-592a945d43ad admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:16.350487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b3071d-0c29-4e9e-92fe-1ff2ecdcec8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:16.351381 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b3071d-0c29-4e9e-92fe-1ff2ecdcec8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:16.351666 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b3071d-0c29-4e9e-92fe-1ff2ecdcec8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:16.351969 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b3071d-0c29-4e9e-92fe-1ff2ecdcec8d admin admin] PATH_INFO: `/v3/projects/a3539359790d4d7d9175d7295c270a19/users/5becb51fc65849cfaa00126a9523e69c/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:16.364828 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:16.368674 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b3071d-0c29-4e9e-92fe-1ff2ecdcec8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3539359790d4d7d9175d7295c270a19, user_id=5becb51fc65849cfaa00126a9523e69c, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:16.372099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b3071d-0c29-4e9e-92fe-1ff2ecdcec8d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:16.386773 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1424/2808] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:51:16 2022] PUT /identity/v3/projects/a3539359790d4d7d9175d7295c270a19/users/5becb51fc65849cfaa00126a9523e69c/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:51:16.398170 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44decf-c18b-4918-aeef-903623a30399 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:16.398566 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44decf-c18b-4918-aeef-903623a30399 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:16.398845 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44decf-c18b-4918-aeef-903623a30399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:16.427394 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-7c44decf-c18b-4918-aeef-903623a30399 None None] MFA Rules not processed for user `5becb51fc65849cfaa00126a9523e69c`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:51:16.428601 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7c44decf-c18b-4918-aeef-903623a30399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:51:16.511642 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1385/2809] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:51:16 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:51:16.522925 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b3071d-0c29-4e9e-92fe-1ff2ecdcec8d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:16.525003 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e6b3071d-0c29-4e9e-92fe-1ff2ecdcec8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:51:16.588695 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b3071d-0c29-4e9e-92fe-1ff2ecdcec8d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:16.592766 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2dfe7b-fd65-48af-ab4a-2f3b9aa34523 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:16.593501 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2dfe7b-fd65-48af-ab4a-2f3b9aa34523 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:16.593876 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2dfe7b-fd65-48af-ab4a-2f3b9aa34523 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:16.594152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2dfe7b-fd65-48af-ab4a-2f3b9aa34523 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:16.603468 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2dfe7b-fd65-48af-ab4a-2f3b9aa34523 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:16.604949 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2dfe7b-fd65-48af-ab4a-2f3b9aa34523 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:16.616659 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1425/2810] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:51:16 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:51:25.049109 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c44decf-c18b-4918-aeef-903623a30399 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:25.103676 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:25.119396 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c44decf-c18b-4918-aeef-903623a30399 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:25.124844 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04904f5f-5877-4b8c-bf90-28d81378fec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:25.125839 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04904f5f-5877-4b8c-bf90-28d81378fec6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:25.126126 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04904f5f-5877-4b8c-bf90-28d81378fec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:25.126426 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04904f5f-5877-4b8c-bf90-28d81378fec6 admin admin] PATH_INFO: `/v3/users/5becb51fc65849cfaa00126a9523e69c` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:25.127631 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04904f5f-5877-4b8c-bf90-28d81378fec6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5becb51fc65849cfaa00126a9523e69c)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:25.129613 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04904f5f-5877-4b8c-bf90-28d81378fec6 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:25.176714 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-04904f5f-5877-4b8c-bf90-28d81378fec6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5becb51fc65849cfaa00126a9523e69c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:51:25.177377 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-04904f5f-5877-4b8c-bf90-28d81378fec6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5becb51fc65849cfaa00126a9523e69c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:51:25.190573 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-04904f5f-5877-4b8c-bf90-28d81378fec6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5becb51fc65849cfaa00126a9523e69c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:51:25.198141 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-04904f5f-5877-4b8c-bf90-28d81378fec6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5becb51fc65849cfaa00126a9523e69c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:51:25.209065 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-04904f5f-5877-4b8c-bf90-28d81378fec6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5becb51fc65849cfaa00126a9523e69c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:51:25.216971 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1386/2811] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:51:25 2022] DELETE /identity/v3/users/5becb51fc65849cfaa00126a9523e69c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:51:25.224190 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f40be9-2b5e-41fc-a34f-d62bec9be329 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:25.225061 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f40be9-2b5e-41fc-a34f-d62bec9be329 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:25.225061 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f40be9-2b5e-41fc-a34f-d62bec9be329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:25.252583 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-65f40be9-2b5e-41fc-a34f-d62bec9be329 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:51:25.254284 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-65f40be9-2b5e-41fc-a34f-d62bec9be329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:51:25.322229 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1426/2812] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:51:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:51:25.333594 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04904f5f-5877-4b8c-bf90-28d81378fec6 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:25.335701 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-04904f5f-5877-4b8c-bf90-28d81378fec6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:51:25.402960 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04904f5f-5877-4b8c-bf90-28d81378fec6 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:25.407083 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d04e446-0228-446d-9723-7ff476807ab8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:25.407770 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d04e446-0228-446d-9723-7ff476807ab8 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:25.408051 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d04e446-0228-446d-9723-7ff476807ab8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:25.408324 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d04e446-0228-446d-9723-7ff476807ab8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:25.418392 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d04e446-0228-446d-9723-7ff476807ab8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:25.419832 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d04e446-0228-446d-9723-7ff476807ab8 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:25.431466 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1387/2813] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:51:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:51:25.728056 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f40be9-2b5e-41fc-a34f-d62bec9be329 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:25.729621 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-65f40be9-2b5e-41fc-a34f-d62bec9be329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:51:25.742662 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f40be9-2b5e-41fc-a34f-d62bec9be329 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:25.748476 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7014994-a64d-450e-91f7-5457672ae367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:25.750494 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7014994-a64d-450e-91f7-5457672ae367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:25.750768 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7014994-a64d-450e-91f7-5457672ae367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:25.751027 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7014994-a64d-450e-91f7-5457672ae367 admin admin] PATH_INFO: `/v3/projects/a3539359790d4d7d9175d7295c270a19` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:25.752412 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7014994-a64d-450e-91f7-5457672ae367 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3539359790d4d7d9175d7295c270a19)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:25.754356 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7014994-a64d-450e-91f7-5457672ae367 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:25.803988 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a7014994-a64d-450e-91f7-5457672ae367 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3539359790d4d7d9175d7295c270a19', 'request_id': 'req-a7014994-a64d-450e-91f7-5457672ae367'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:51:25.810680 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a7014994-a64d-450e-91f7-5457672ae367 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3539359790d4d7d9175d7295c270a19', 'request_id': 'req-a7014994-a64d-450e-91f7-5457672ae367'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:51:25.816922 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a7014994-a64d-450e-91f7-5457672ae367 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3539359790d4d7d9175d7295c270a19', 'request_id': 'req-a7014994-a64d-450e-91f7-5457672ae367'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:51:25.831758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a7014994-a64d-450e-91f7-5457672ae367 admin admin] The token cache is being invalidate because project a3539359790d4d7d9175d7295c270a19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:51:25.832044 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a7014994-a64d-450e-91f7-5457672ae367 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:51:25.832658 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a7014994-a64d-450e-91f7-5457672ae367 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:51:25.834253 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1427/2814] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:51:25 2022] DELETE /identity/v3/projects/a3539359790d4d7d9175d7295c270a19 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:51:25.839435 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56430970-626a-4458-8242-a94720dd26ec None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:25.839720 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56430970-626a-4458-8242-a94720dd26ec None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:25.839997 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56430970-626a-4458-8242-a94720dd26ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:25.863305 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-56430970-626a-4458-8242-a94720dd26ec None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:51:25.864566 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-56430970-626a-4458-8242-a94720dd26ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:51:25.937981 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1388/2815] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:51:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:51:25.946728 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd01873e-cca9-4db3-9edd-0708f0938877 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:25.947236 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd01873e-cca9-4db3-9edd-0708f0938877 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:25.947709 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd01873e-cca9-4db3-9edd-0708f0938877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:25.973660 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-fd01873e-cca9-4db3-9edd-0708f0938877 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:51:25.975641 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fd01873e-cca9-4db3-9edd-0708f0938877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:51:25.989353 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1428/2816] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:51:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:51:25.995703 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56430970-626a-4458-8242-a94720dd26ec None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:26.006257 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56430970-626a-4458-8242-a94720dd26ec None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:26.011935 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d58a80f-51d4-4918-923d-d45e0d652457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:26.012607 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58a80f-51d4-4918-923d-d45e0d652457 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:26.012895 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58a80f-51d4-4918-923d-d45e0d652457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:26.013175 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58a80f-51d4-4918-923d-d45e0d652457 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:26.013914 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d58a80f-51d4-4918-923d-d45e0d652457 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:51:26.014226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d58a80f-51d4-4918-923d-d45e0d652457 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:26.015890 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d58a80f-51d4-4918-923d-d45e0d652457 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:26.022768 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:26.036685 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1389/2817] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:51:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:51:26.037974 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd01873e-cca9-4db3-9edd-0708f0938877 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:26.047828 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd01873e-cca9-4db3-9edd-0708f0938877 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:26.053398 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb23b85-00e5-440c-a6af-56420088d9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:26.054070 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb23b85-00e5-440c-a6af-56420088d9f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:26.054391 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb23b85-00e5-440c-a6af-56420088d9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:26.054669 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb23b85-00e5-440c-a6af-56420088d9f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:26.055368 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb23b85-00e5-440c-a6af-56420088d9f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:26.057304 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb23b85-00e5-440c-a6af-56420088d9f8 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:26.066035 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:26.086328 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1429/2818] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:51:26 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:51:26.091193 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d58a80f-51d4-4918-923d-d45e0d652457 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:26.101606 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d58a80f-51d4-4918-923d-d45e0d652457 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:26.164062 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d42587-cd09-44e3-9bbc-d9cb4a2d0305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:26.164736 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d42587-cd09-44e3-9bbc-d9cb4a2d0305 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:26.165017 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d42587-cd09-44e3-9bbc-d9cb4a2d0305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:26.165290 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d42587-cd09-44e3-9bbc-d9cb4a2d0305 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:26.165978 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d42587-cd09-44e3-9bbc-d9cb4a2d0305 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:26.167928 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d42587-cd09-44e3-9bbc-d9cb4a2d0305 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:26.190538 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1390/2819] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:51:26 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:51:26.195145 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb23b85-00e5-440c-a6af-56420088d9f8 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:26.210392 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb23b85-00e5-440c-a6af-56420088d9f8 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:26.223086 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b8ed1-a3ff-4d18-8580-e9e641abef11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:26.223793 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b8ed1-a3ff-4d18-8580-e9e641abef11 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:26.224073 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b8ed1-a3ff-4d18-8580-e9e641abef11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:26.224349 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b8ed1-a3ff-4d18-8580-e9e641abef11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:26.225094 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23b8ed1-a3ff-4d18-8580-e9e641abef11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:26.226758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23b8ed1-a3ff-4d18-8580-e9e641abef11 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:26.237921 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1430/2820] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:51:26 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:51:26.242480 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d42587-cd09-44e3-9bbc-d9cb4a2d0305 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:26.254200 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d42587-cd09-44e3-9bbc-d9cb4a2d0305 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:26.258925 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79762c1-b152-4b27-8531-f7b288851d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:26.259592 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79762c1-b152-4b27-8531-f7b288851d7a admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:26.259871 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79762c1-b152-4b27-8531-f7b288851d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:26.260172 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79762c1-b152-4b27-8531-f7b288851d7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:26.261961 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79762c1-b152-4b27-8531-f7b288851d7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:26.266651 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79762c1-b152-4b27-8531-f7b288851d7a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:26.278377 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1391/2821] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:51:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:51:26.282964 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b8ed1-a3ff-4d18-8580-e9e641abef11 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:26.294532 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b8ed1-a3ff-4d18-8580-e9e641abef11 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:26.300399 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a05758c-5e2f-4ce2-ad4f-0d84ae1f5cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:26.301313 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05758c-5e2f-4ce2-ad4f-0d84ae1f5cc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:26.301598 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05758c-5e2f-4ce2-ad4f-0d84ae1f5cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:26.301880 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05758c-5e2f-4ce2-ad4f-0d84ae1f5cc6 admin admin] PATH_INFO: `/v3/projects/9222efc77d154b5d88c99170b91e0cae/users/ba81e18a0ff64dbe90d5534e3e25193e/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:26.318022 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a05758c-5e2f-4ce2-ad4f-0d84ae1f5cc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9222efc77d154b5d88c99170b91e0cae, user_id=ba81e18a0ff64dbe90d5534e3e25193e, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:26.320845 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a05758c-5e2f-4ce2-ad4f-0d84ae1f5cc6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:26.337958 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1431/2822] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:51:26 2022] PUT /identity/v3/projects/9222efc77d154b5d88c99170b91e0cae/users/ba81e18a0ff64dbe90d5534e3e25193e/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:51:26.348955 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95317cd-636e-42fb-acff-95078cdeb42f None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:26.349245 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95317cd-636e-42fb-acff-95078cdeb42f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:26.349520 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95317cd-636e-42fb-acff-95078cdeb42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:26.377670 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-a95317cd-636e-42fb-acff-95078cdeb42f None None] MFA Rules not processed for user `ba81e18a0ff64dbe90d5534e3e25193e`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:51:26.378942 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a95317cd-636e-42fb-acff-95078cdeb42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:51:26.453664 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1392/2823] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:51:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:51:26.460393 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a05758c-5e2f-4ce2-ad4f-0d84ae1f5cc6 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:26.532669 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a05758c-5e2f-4ce2-ad4f-0d84ae1f5cc6 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:26.537945 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac5610-4ca4-4c6e-aa64-97ef82e1f219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:26.538648 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac5610-4ca4-4c6e-aa64-97ef82e1f219 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:26.538937 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac5610-4ca4-4c6e-aa64-97ef82e1f219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:26.539214 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac5610-4ca4-4c6e-aa64-97ef82e1f219 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:26.539911 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eac5610-4ca4-4c6e-aa64-97ef82e1f219 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:26.541835 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eac5610-4ca4-4c6e-aa64-97ef82e1f219 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:26.574920 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1432/2824] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:51:26 2022] POST /identity/v3/projects => generated 373 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:51:26.580153 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95317cd-636e-42fb-acff-95078cdeb42f None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:26.598144 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95317cd-636e-42fb-acff-95078cdeb42f None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:26.664511 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9894236-f2eb-4f69-aa5c-e6c27b3e00e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:26.665186 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9894236-f2eb-4f69-aa5c-e6c27b3e00e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:26.665465 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9894236-f2eb-4f69-aa5c-e6c27b3e00e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:26.665729 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9894236-f2eb-4f69-aa5c-e6c27b3e00e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:26.666458 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9894236-f2eb-4f69-aa5c-e6c27b3e00e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:26.668508 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9894236-f2eb-4f69-aa5c-e6c27b3e00e0 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:26.678379 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:26.692099 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1393/2825] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:51:26 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:51:26.696196 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac5610-4ca4-4c6e-aa64-97ef82e1f219 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:26.708573 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac5610-4ca4-4c6e-aa64-97ef82e1f219 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:26.714105 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe0a3ca-8a8a-462f-9fc2-6801a293c849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:26.714813 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe0a3ca-8a8a-462f-9fc2-6801a293c849 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:26.715143 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe0a3ca-8a8a-462f-9fc2-6801a293c849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:26.715376 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe0a3ca-8a8a-462f-9fc2-6801a293c849 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:26.716105 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe0a3ca-8a8a-462f-9fc2-6801a293c849 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:26.717746 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe0a3ca-8a8a-462f-9fc2-6801a293c849 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:26.728921 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1433/2826] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:51:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:51:26.734960 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9894236-f2eb-4f69-aa5c-e6c27b3e00e0 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:26.747548 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9894236-f2eb-4f69-aa5c-e6c27b3e00e0 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:26.752845 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe975cd-e152-40ff-86bb-6f5f71a0f820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:26.753769 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe975cd-e152-40ff-86bb-6f5f71a0f820 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:26.754046 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe975cd-e152-40ff-86bb-6f5f71a0f820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:26.754356 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe975cd-e152-40ff-86bb-6f5f71a0f820 admin admin] PATH_INFO: `/v3/projects/eba29dc21588471987becfd397c4d212/users/0cb36186710141eb8065e9fe75114e82/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:26.770482 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe975cd-e152-40ff-86bb-6f5f71a0f820 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eba29dc21588471987becfd397c4d212, user_id=0cb36186710141eb8065e9fe75114e82, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:26.773183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe975cd-e152-40ff-86bb-6f5f71a0f820 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:26.790691 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1394/2827] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:51:26 2022] PUT /identity/v3/projects/eba29dc21588471987becfd397c4d212/users/0cb36186710141eb8065e9fe75114e82/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:51:26.796788 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe0a3ca-8a8a-462f-9fc2-6801a293c849 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:26.810089 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe0a3ca-8a8a-462f-9fc2-6801a293c849 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:26.869839 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf9c11-9bd8-40e9-90fe-8d818a56135e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:26.870641 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf9c11-9bd8-40e9-90fe-8d818a56135e admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:26.871010 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf9c11-9bd8-40e9-90fe-8d818a56135e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:26.871287 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf9c11-9bd8-40e9-90fe-8d818a56135e admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:26.872022 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddf9c11-9bd8-40e9-90fe-8d818a56135e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:26.873630 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddf9c11-9bd8-40e9-90fe-8d818a56135e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:26.883628 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1434/2828] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:51:26 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:51:26.890528 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe975cd-e152-40ff-86bb-6f5f71a0f820 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:26.905463 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe975cd-e152-40ff-86bb-6f5f71a0f820 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:26.911577 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4657e36-9865-4c6c-b7b8-b8e7f941e0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:26.912476 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4657e36-9865-4c6c-b7b8-b8e7f941e0c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:26.912752 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4657e36-9865-4c6c-b7b8-b8e7f941e0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:26.913022 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4657e36-9865-4c6c-b7b8-b8e7f941e0c1 admin admin] PATH_INFO: `/v3/projects/eba29dc21588471987becfd397c4d212/users/0cb36186710141eb8065e9fe75114e82/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:26.915292 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4657e36-9865-4c6c-b7b8-b8e7f941e0c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eba29dc21588471987becfd397c4d212, user_id=0cb36186710141eb8065e9fe75114e82, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:26.918399 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4657e36-9865-4c6c-b7b8-b8e7f941e0c1 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:26.929598 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1395/2829] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:51:26 2022] PUT /identity/v3/projects/eba29dc21588471987becfd397c4d212/users/0cb36186710141eb8065e9fe75114e82/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:51:26.941637 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344049d-6e89-4282-aa66-2b78af8b7374 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:26.941925 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344049d-6e89-4282-aa66-2b78af8b7374 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:26.942202 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344049d-6e89-4282-aa66-2b78af8b7374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:26.971295 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-4344049d-6e89-4282-aa66-2b78af8b7374 None None] MFA Rules not processed for user `0cb36186710141eb8065e9fe75114e82`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:51:26.972434 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4344049d-6e89-4282-aa66-2b78af8b7374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:51:26.989468 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:27.050455 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1435/2830] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:51:26 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:51:27.062716 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4657e36-9865-4c6c-b7b8-b8e7f941e0c1 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:27.064982 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c4657e36-9865-4c6c-b7b8-b8e7f941e0c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:51:27.127170 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4657e36-9865-4c6c-b7b8-b8e7f941e0c1 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:27.131251 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00569d23-db21-4cd6-8ad9-993759dddc54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:27.131906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00569d23-db21-4cd6-8ad9-993759dddc54 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:27.132187 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00569d23-db21-4cd6-8ad9-993759dddc54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:27.132468 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00569d23-db21-4cd6-8ad9-993759dddc54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:27.142755 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00569d23-db21-4cd6-8ad9-993759dddc54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:27.144195 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00569d23-db21-4cd6-8ad9-993759dddc54 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:27.154385 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1396/2831] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:51:27 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:51:27.233226 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4344049d-6e89-4282-aa66-2b78af8b7374 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:51:27.244478 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4344049d-6e89-4282-aa66-2b78af8b7374 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:51:27.249528 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3eff08-dc84-4db7-af66-e9e3c436a1be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:51:27.250176 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3eff08-dc84-4db7-af66-e9e3c436a1be service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:51:27.250487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3eff08-dc84-4db7-af66-e9e3c436a1be service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:51:27.250760 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3eff08-dc84-4db7-af66-e9e3c436a1be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:51:27.320888 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3eff08-dc84-4db7-af66-e9e3c436a1be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:51:27.322428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3eff08-dc84-4db7-af66-e9e3c436a1be service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:51:27.332520 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1436/2832] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:51:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:51:37.149737 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=43,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:51:37.327647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=51 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:52:09.503139 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00569d23-db21-4cd6-8ad9-993759dddc54 service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:09.568325 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00569d23-db21-4cd6-8ad9-993759dddc54 service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:09.574081 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d159399e-68bb-4d98-b36b-dc9e144f83d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:09.574994 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159399e-68bb-4d98-b36b-dc9e144f83d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:09.575663 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159399e-68bb-4d98-b36b-dc9e144f83d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:09.575953 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159399e-68bb-4d98-b36b-dc9e144f83d5 admin admin] PATH_INFO: `/v3/users/ba81e18a0ff64dbe90d5534e3e25193e` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:09.577183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d159399e-68bb-4d98-b36b-dc9e144f83d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba81e18a0ff64dbe90d5534e3e25193e)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:09.579184 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d159399e-68bb-4d98-b36b-dc9e144f83d5 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:09.637802 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d159399e-68bb-4d98-b36b-dc9e144f83d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba81e18a0ff64dbe90d5534e3e25193e'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:09.638501 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d159399e-68bb-4d98-b36b-dc9e144f83d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba81e18a0ff64dbe90d5534e3e25193e'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:09.656683 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d159399e-68bb-4d98-b36b-dc9e144f83d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba81e18a0ff64dbe90d5534e3e25193e'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:09.666474 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d159399e-68bb-4d98-b36b-dc9e144f83d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba81e18a0ff64dbe90d5534e3e25193e'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:09.677296 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d159399e-68bb-4d98-b36b-dc9e144f83d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba81e18a0ff64dbe90d5534e3e25193e'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:09.686402 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1397/2833] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:52:09 2022] DELETE /identity/v3/users/ba81e18a0ff64dbe90d5534e3e25193e => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:52:09.691291 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b52087-3ba3-4acd-832b-691b13d6690e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:09.691827 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b52087-3ba3-4acd-832b-691b13d6690e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:09.691827 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b52087-3ba3-4acd-832b-691b13d6690e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:09.712720 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e4b52087-3ba3-4acd-832b-691b13d6690e None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:52:09.714678 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e4b52087-3ba3-4acd-832b-691b13d6690e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:09.782148 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1437/2834] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:52:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:52:09.792441 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d159399e-68bb-4d98-b36b-dc9e144f83d5 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:09.794136 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d159399e-68bb-4d98-b36b-dc9e144f83d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:09.859264 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d159399e-68bb-4d98-b36b-dc9e144f83d5 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:09.863235 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376aea5-3006-4f37-8a54-9a6f8433a5da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:09.863896 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376aea5-3006-4f37-8a54-9a6f8433a5da service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:09.864180 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376aea5-3006-4f37-8a54-9a6f8433a5da service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:09.864455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376aea5-3006-4f37-8a54-9a6f8433a5da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:09.875183 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3376aea5-3006-4f37-8a54-9a6f8433a5da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:09.876627 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3376aea5-3006-4f37-8a54-9a6f8433a5da service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:09.886958 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1398/2835] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:52:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:52:10.176985 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b52087-3ba3-4acd-832b-691b13d6690e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:10.178536 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e4b52087-3ba3-4acd-832b-691b13d6690e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:10.191110 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b52087-3ba3-4acd-832b-691b13d6690e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:10.198678 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e709d61-6017-45d1-a766-e6de3a62b3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:10.199627 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e709d61-6017-45d1-a766-e6de3a62b3d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:10.199875 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e709d61-6017-45d1-a766-e6de3a62b3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:10.200230 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e709d61-6017-45d1-a766-e6de3a62b3d5 admin admin] PATH_INFO: `/v3/projects/9222efc77d154b5d88c99170b91e0cae` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:10.201350 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e709d61-6017-45d1-a766-e6de3a62b3d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9222efc77d154b5d88c99170b91e0cae)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:10.203380 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e709d61-6017-45d1-a766-e6de3a62b3d5 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:10.260021 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0e709d61-6017-45d1-a766-e6de3a62b3d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9222efc77d154b5d88c99170b91e0cae', 'request_id': 'req-0e709d61-6017-45d1-a766-e6de3a62b3d5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:10.266213 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0e709d61-6017-45d1-a766-e6de3a62b3d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9222efc77d154b5d88c99170b91e0cae', 'request_id': 'req-0e709d61-6017-45d1-a766-e6de3a62b3d5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:10.271913 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0e709d61-6017-45d1-a766-e6de3a62b3d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9222efc77d154b5d88c99170b91e0cae', 'request_id': 'req-0e709d61-6017-45d1-a766-e6de3a62b3d5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:10.287276 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0e709d61-6017-45d1-a766-e6de3a62b3d5 admin admin] The token cache is being invalidate because project 9222efc77d154b5d88c99170b91e0cae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:52:10.287564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0e709d61-6017-45d1-a766-e6de3a62b3d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:10.288186 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0e709d61-6017-45d1-a766-e6de3a62b3d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:10.289883 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1438/2836] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:52:10 2022] DELETE /identity/v3/projects/9222efc77d154b5d88c99170b91e0cae => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:52:10.296182 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376aea5-3006-4f37-8a54-9a6f8433a5da service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:10.308651 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376aea5-3006-4f37-8a54-9a6f8433a5da service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:10.327267 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:52:10.372345 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef85883-9c21-4192-818e-93a63d451050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:10.373536 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef85883-9c21-4192-818e-93a63d451050 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:10.373774 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef85883-9c21-4192-818e-93a63d451050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:10.374101 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef85883-9c21-4192-818e-93a63d451050 admin admin] PATH_INFO: `/v3/users/0cb36186710141eb8065e9fe75114e82` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:10.376061 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef85883-9c21-4192-818e-93a63d451050 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cb36186710141eb8065e9fe75114e82)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:10.378853 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef85883-9c21-4192-818e-93a63d451050 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:10.432474 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6ef85883-9c21-4192-818e-93a63d451050 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cb36186710141eb8065e9fe75114e82'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:10.433257 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6ef85883-9c21-4192-818e-93a63d451050 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cb36186710141eb8065e9fe75114e82'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:10.450956 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6ef85883-9c21-4192-818e-93a63d451050 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cb36186710141eb8065e9fe75114e82'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:10.459397 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6ef85883-9c21-4192-818e-93a63d451050 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cb36186710141eb8065e9fe75114e82'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:10.470137 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6ef85883-9c21-4192-818e-93a63d451050 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cb36186710141eb8065e9fe75114e82'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:10.477934 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1399/2837] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:52:10 2022] DELETE /identity/v3/users/0cb36186710141eb8065e9fe75114e82 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:52:10.767091 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e709d61-6017-45d1-a766-e6de3a62b3d5 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:10.768555 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0e709d61-6017-45d1-a766-e6de3a62b3d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:10.785592 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e709d61-6017-45d1-a766-e6de3a62b3d5 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:10.847997 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ebc69-4d9d-4123-868d-44fea694a667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:10.849052 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ebc69-4d9d-4123-868d-44fea694a667 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:10.849422 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ebc69-4d9d-4123-868d-44fea694a667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:10.849698 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ebc69-4d9d-4123-868d-44fea694a667 admin admin] PATH_INFO: `/v3/projects/eba29dc21588471987becfd397c4d212` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:10.851337 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ebc69-4d9d-4123-868d-44fea694a667 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eba29dc21588471987becfd397c4d212)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:10.854012 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ebc69-4d9d-4123-868d-44fea694a667 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:10.889114 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:52:10.907208 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-844ebc69-4d9d-4123-868d-44fea694a667 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eba29dc21588471987becfd397c4d212', 'request_id': 'req-844ebc69-4d9d-4123-868d-44fea694a667'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:10.913369 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-844ebc69-4d9d-4123-868d-44fea694a667 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eba29dc21588471987becfd397c4d212', 'request_id': 'req-844ebc69-4d9d-4123-868d-44fea694a667'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:10.919759 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-844ebc69-4d9d-4123-868d-44fea694a667 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eba29dc21588471987becfd397c4d212', 'request_id': 'req-844ebc69-4d9d-4123-868d-44fea694a667'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:10.935555 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-844ebc69-4d9d-4123-868d-44fea694a667 admin admin] The token cache is being invalidate because project eba29dc21588471987becfd397c4d212 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:52:10.935778 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-844ebc69-4d9d-4123-868d-44fea694a667 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:10.936710 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-844ebc69-4d9d-4123-868d-44fea694a667 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:10.939084 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1439/2838] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:52:10 2022] DELETE /identity/v3/projects/eba29dc21588471987becfd397c4d212 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:52:10.945562 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2b8ea-4084-4858-aae5-5a20c892d126 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:10.946053 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2b8ea-4084-4858-aae5-5a20c892d126 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:10.946053 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2b8ea-4084-4858-aae5-5a20c892d126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:10.965909 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-5cb2b8ea-4084-4858-aae5-5a20c892d126 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:52:10.967162 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5cb2b8ea-4084-4858-aae5-5a20c892d126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:11.032882 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1400/2839] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:52:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:52:11.041181 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8016f286-e0ae-4c9d-9b01-d4c1cc65b695 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:11.041471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8016f286-e0ae-4c9d-9b01-d4c1cc65b695 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:11.041751 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8016f286-e0ae-4c9d-9b01-d4c1cc65b695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:11.065564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-8016f286-e0ae-4c9d-9b01-d4c1cc65b695 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:52:11.066836 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8016f286-e0ae-4c9d-9b01-d4c1cc65b695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:11.075595 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1440/2840] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:52:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:52:11.080444 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb2b8ea-4084-4858-aae5-5a20c892d126 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:11.091565 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb2b8ea-4084-4858-aae5-5a20c892d126 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:11.099395 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838da5f2-d732-4e45-b824-6014f4992e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:11.100384 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838da5f2-d732-4e45-b824-6014f4992e39 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:11.100794 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838da5f2-d732-4e45-b824-6014f4992e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:11.101164 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838da5f2-d732-4e45-b824-6014f4992e39 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:11.102150 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838da5f2-d732-4e45-b824-6014f4992e39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:52:11.102515 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838da5f2-d732-4e45-b824-6014f4992e39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:11.104123 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838da5f2-d732-4e45-b824-6014f4992e39 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:11.116086 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1401/2841] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:52:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:52:11.121343 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8016f286-e0ae-4c9d-9b01-d4c1cc65b695 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:11.131047 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8016f286-e0ae-4c9d-9b01-d4c1cc65b695 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:11.136555 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8fba08-a153-4546-8b63-df8568db9051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:11.137208 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fba08-a153-4546-8b63-df8568db9051 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:11.137487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fba08-a153-4546-8b63-df8568db9051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:11.137736 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fba08-a153-4546-8b63-df8568db9051 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:11.138790 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8fba08-a153-4546-8b63-df8568db9051 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:11.140896 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8fba08-a153-4546-8b63-df8568db9051 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:11.174907 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1441/2842] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:52:11 2022] POST /identity/v3/projects => generated 363 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:52:11.181736 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838da5f2-d732-4e45-b824-6014f4992e39 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:11.197033 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838da5f2-d732-4e45-b824-6014f4992e39 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:11.225101 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:52:11.261973 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116dfc52-e1f7-4dc4-a5ea-257cfa5edbcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:11.262707 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116dfc52-e1f7-4dc4-a5ea-257cfa5edbcb admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:11.263005 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116dfc52-e1f7-4dc4-a5ea-257cfa5edbcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:11.263284 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116dfc52-e1f7-4dc4-a5ea-257cfa5edbcb admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:11.263984 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116dfc52-e1f7-4dc4-a5ea-257cfa5edbcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:11.265930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116dfc52-e1f7-4dc4-a5ea-257cfa5edbcb admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:11.289731 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1402/2843] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:52:11 2022] POST /identity/v3/users => generated 282 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:52:11.294145 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8fba08-a153-4546-8b63-df8568db9051 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:11.305734 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8fba08-a153-4546-8b63-df8568db9051 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:11.311536 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d108109-bbdc-43d6-bf49-78c5b67ea331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:11.312217 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d108109-bbdc-43d6-bf49-78c5b67ea331 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:11.312500 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d108109-bbdc-43d6-bf49-78c5b67ea331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:11.312782 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d108109-bbdc-43d6-bf49-78c5b67ea331 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:11.313543 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d108109-bbdc-43d6-bf49-78c5b67ea331 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:11.315204 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d108109-bbdc-43d6-bf49-78c5b67ea331 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:11.326758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1442/2844] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:52:11.332030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116dfc52-e1f7-4dc4-a5ea-257cfa5edbcb admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:11.348435 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116dfc52-e1f7-4dc4-a5ea-257cfa5edbcb admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:11.355631 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a4c69-a493-4524-abc1-a65807617c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:11.356310 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a4c69-a493-4524-abc1-a65807617c43 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:11.356596 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a4c69-a493-4524-abc1-a65807617c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:11.356875 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a4c69-a493-4524-abc1-a65807617c43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:11.357610 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a4c69-a493-4524-abc1-a65807617c43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:11.359255 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a4c69-a493-4524-abc1-a65807617c43 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:11.370839 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1403/2845] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:52:11.376844 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d108109-bbdc-43d6-bf49-78c5b67ea331 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:11.389010 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d108109-bbdc-43d6-bf49-78c5b67ea331 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:11.394121 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bdec49-4c65-4d4a-9118-cc07ee386b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:11.395057 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bdec49-4c65-4d4a-9118-cc07ee386b6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:11.395345 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bdec49-4c65-4d4a-9118-cc07ee386b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:11.395617 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bdec49-4c65-4d4a-9118-cc07ee386b6c admin admin] PATH_INFO: `/v3/projects/06f62aa93e31427db87c39bcddee88b6/users/a1ab2f1290604293bc2c6059c4ba9d23/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:11.411604 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bdec49-4c65-4d4a-9118-cc07ee386b6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=06f62aa93e31427db87c39bcddee88b6, user_id=a1ab2f1290604293bc2c6059c4ba9d23, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:11.414425 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bdec49-4c65-4d4a-9118-cc07ee386b6c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:11.428835 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1443/2846] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:52:11 2022] PUT /identity/v3/projects/06f62aa93e31427db87c39bcddee88b6/users/a1ab2f1290604293bc2c6059c4ba9d23/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:52:11.439818 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ee601-4d07-42cb-b839-b31871405bb9 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:11.440111 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ee601-4d07-42cb-b839-b31871405bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:11.440388 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ee601-4d07-42cb-b839-b31871405bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:11.468446 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-850ee601-4d07-42cb-b839-b31871405bb9 None None] MFA Rules not processed for user `a1ab2f1290604293bc2c6059c4ba9d23`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:52:11.469551 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-850ee601-4d07-42cb-b839-b31871405bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:11.553686 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1404/2847] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:52:11 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:52:11.562718 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bdec49-4c65-4d4a-9118-cc07ee386b6c admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:11.628779 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bdec49-4c65-4d4a-9118-cc07ee386b6c admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:11.634960 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda92d1-af7d-4a69-a89e-c3c84e736639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:11.635642 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda92d1-af7d-4a69-a89e-c3c84e736639 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:11.635897 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda92d1-af7d-4a69-a89e-c3c84e736639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:11.636163 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda92d1-af7d-4a69-a89e-c3c84e736639 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:11.636854 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda92d1-af7d-4a69-a89e-c3c84e736639 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:11.638873 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda92d1-af7d-4a69-a89e-c3c84e736639 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:11.669312 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1444/2848] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:52:11 2022] POST /identity/v3/projects => generated 365 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:52:11.674179 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ee601-4d07-42cb-b839-b31871405bb9 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:11.686332 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ee601-4d07-42cb-b839-b31871405bb9 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:11.752096 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbe9bbb-bf4e-4d37-8dc9-4c389b79917f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:11.752881 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe9bbb-bf4e-4d37-8dc9-4c389b79917f admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:11.753279 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe9bbb-bf4e-4d37-8dc9-4c389b79917f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:11.753634 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe9bbb-bf4e-4d37-8dc9-4c389b79917f admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:11.754651 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbe9bbb-bf4e-4d37-8dc9-4c389b79917f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:11.757605 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbe9bbb-bf4e-4d37-8dc9-4c389b79917f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:11.783325 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1405/2849] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:52:11 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 21:52:11.787526 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda92d1-af7d-4a69-a89e-c3c84e736639 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:11.799052 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda92d1-af7d-4a69-a89e-c3c84e736639 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:11.804282 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef73d2-a411-46db-812f-42f4c8c56c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:11.805201 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef73d2-a411-46db-812f-42f4c8c56c9f admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:11.805354 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef73d2-a411-46db-812f-42f4c8c56c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:11.805605 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef73d2-a411-46db-812f-42f4c8c56c9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:11.806361 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ef73d2-a411-46db-812f-42f4c8c56c9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:11.807961 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ef73d2-a411-46db-812f-42f4c8c56c9f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:11.819456 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1445/2850] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:52:11.824069 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbe9bbb-bf4e-4d37-8dc9-4c389b79917f admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:11.837884 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbe9bbb-bf4e-4d37-8dc9-4c389b79917f admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:11.843974 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c8718e-f9cb-4abd-9fe0-f0d36d071e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:11.844925 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c8718e-f9cb-4abd-9fe0-f0d36d071e88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:11.845208 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c8718e-f9cb-4abd-9fe0-f0d36d071e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:11.845486 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c8718e-f9cb-4abd-9fe0-f0d36d071e88 admin admin] PATH_INFO: `/v3/projects/897b681b309d4f5c998c1ce57ebbbae7/users/6166167ecb7049388db0349a19456281/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:11.863271 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c8718e-f9cb-4abd-9fe0-f0d36d071e88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=897b681b309d4f5c998c1ce57ebbbae7, user_id=6166167ecb7049388db0349a19456281, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:11.866014 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c8718e-f9cb-4abd-9fe0-f0d36d071e88 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:11.885594 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1406/2851] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:52:11 2022] PUT /identity/v3/projects/897b681b309d4f5c998c1ce57ebbbae7/users/6166167ecb7049388db0349a19456281/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:52:11.889972 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef73d2-a411-46db-812f-42f4c8c56c9f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:11.901849 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef73d2-a411-46db-812f-42f4c8c56c9f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:11.922264 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:52:11.972010 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44886a97-6c8d-4120-8982-040c32f2f765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:11.972701 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44886a97-6c8d-4120-8982-040c32f2f765 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:11.972988 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44886a97-6c8d-4120-8982-040c32f2f765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:11.973265 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44886a97-6c8d-4120-8982-040c32f2f765 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:11.973976 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44886a97-6c8d-4120-8982-040c32f2f765 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:11.975620 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44886a97-6c8d-4120-8982-040c32f2f765 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:11.987056 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1446/2852] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:52:11.991549 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c8718e-f9cb-4abd-9fe0-f0d36d071e88 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:11.998206 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:52:12.005212 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c8718e-f9cb-4abd-9fe0-f0d36d071e88 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:12.011411 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d2ee8-86b4-434d-a92c-f49f1ab210b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:12.012357 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d2ee8-86b4-434d-a92c-f49f1ab210b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:12.012634 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d2ee8-86b4-434d-a92c-f49f1ab210b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:12.012909 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d2ee8-86b4-434d-a92c-f49f1ab210b0 admin admin] PATH_INFO: `/v3/projects/897b681b309d4f5c998c1ce57ebbbae7/users/6166167ecb7049388db0349a19456281/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:12.014967 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d2ee8-86b4-434d-a92c-f49f1ab210b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=897b681b309d4f5c998c1ce57ebbbae7, user_id=6166167ecb7049388db0349a19456281, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:12.017595 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d2ee8-86b4-434d-a92c-f49f1ab210b0 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:12.028291 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1407/2853] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:52:11 2022] PUT /identity/v3/projects/897b681b309d4f5c998c1ce57ebbbae7/users/6166167ecb7049388db0349a19456281/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:52:12.038744 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab50bc-2d0e-41d4-b321-d31d74394755 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:12.039113 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab50bc-2d0e-41d4-b321-d31d74394755 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:12.039482 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab50bc-2d0e-41d4-b321-d31d74394755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:12.068370 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-1aab50bc-2d0e-41d4-b321-d31d74394755 None None] MFA Rules not processed for user `6166167ecb7049388db0349a19456281`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:52:12.069571 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1aab50bc-2d0e-41d4-b321-d31d74394755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:12.140684 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1447/2854] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:52:12 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:52:12.152193 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d2ee8-86b4-434d-a92c-f49f1ab210b0 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:12.154068 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-777d2ee8-86b4-434d-a92c-f49f1ab210b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:12.219407 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d2ee8-86b4-434d-a92c-f49f1ab210b0 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:12.223472 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea556b-e975-48ff-9b1b-67213c64403a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:12.224137 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea556b-e975-48ff-9b1b-67213c64403a service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:12.224414 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea556b-e975-48ff-9b1b-67213c64403a service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:12.224687 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea556b-e975-48ff-9b1b-67213c64403a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:12.233875 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dea556b-e975-48ff-9b1b-67213c64403a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:12.235343 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dea556b-e975-48ff-9b1b-67213c64403a service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:12.244830 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1408/2855] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:52:12 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:52:12.330275 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab50bc-2d0e-41d4-b321-d31d74394755 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:12.341122 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab50bc-2d0e-41d4-b321-d31d74394755 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:12.345736 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae2a74-debb-4710-98a0-6226c2d519c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:12.346915 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae2a74-debb-4710-98a0-6226c2d519c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:12.347416 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae2a74-debb-4710-98a0-6226c2d519c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:12.347799 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae2a74-debb-4710-98a0-6226c2d519c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:12.420894 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae2a74-debb-4710-98a0-6226c2d519c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:12.422397 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae2a74-debb-4710-98a0-6226c2d519c3 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:12.431798 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1448/2856] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:52:12 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:52:15.399106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea556b-e975-48ff-9b1b-67213c64403a service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:15.465492 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea556b-e975-48ff-9b1b-67213c64403a service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:15.470941 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008efe40-c999-43b2-9057-e58e21c3edcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:15.471817 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008efe40-c999-43b2-9057-e58e21c3edcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:15.472094 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008efe40-c999-43b2-9057-e58e21c3edcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:15.472379 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008efe40-c999-43b2-9057-e58e21c3edcc admin admin] PATH_INFO: `/v3/users/a1ab2f1290604293bc2c6059c4ba9d23` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:15.473664 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008efe40-c999-43b2-9057-e58e21c3edcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1ab2f1290604293bc2c6059c4ba9d23)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:15.475623 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008efe40-c999-43b2-9057-e58e21c3edcc admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:15.525300 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-008efe40-c999-43b2-9057-e58e21c3edcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1ab2f1290604293bc2c6059c4ba9d23'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:15.526077 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-008efe40-c999-43b2-9057-e58e21c3edcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1ab2f1290604293bc2c6059c4ba9d23'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:15.539963 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-008efe40-c999-43b2-9057-e58e21c3edcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1ab2f1290604293bc2c6059c4ba9d23'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:15.557748 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-008efe40-c999-43b2-9057-e58e21c3edcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1ab2f1290604293bc2c6059c4ba9d23'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:15.569138 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-008efe40-c999-43b2-9057-e58e21c3edcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1ab2f1290604293bc2c6059c4ba9d23'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:15.576222 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1409/2857] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:52:15 2022] DELETE /identity/v3/users/a1ab2f1290604293bc2c6059c4ba9d23 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:52:15.581485 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b495c4-3582-4410-870d-5f2906e0867a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:15.582026 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b495c4-3582-4410-870d-5f2906e0867a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:15.582026 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b495c4-3582-4410-870d-5f2906e0867a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:15.600733 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d6b495c4-3582-4410-870d-5f2906e0867a None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:52:15.601985 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d6b495c4-3582-4410-870d-5f2906e0867a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:15.646836 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:52:15.667860 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1449/2858] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:52:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:52:15.677769 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008efe40-c999-43b2-9057-e58e21c3edcc admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:15.679404 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-008efe40-c999-43b2-9057-e58e21c3edcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:15.695152 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:52:15.744605 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008efe40-c999-43b2-9057-e58e21c3edcc admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:15.751427 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7468c2-e171-4832-89c6-c74056f80e52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:15.752382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7468c2-e171-4832-89c6-c74056f80e52 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:15.752785 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7468c2-e171-4832-89c6-c74056f80e52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:15.753123 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7468c2-e171-4832-89c6-c74056f80e52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:15.766339 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7468c2-e171-4832-89c6-c74056f80e52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:15.767798 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7468c2-e171-4832-89c6-c74056f80e52 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:15.778269 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1410/2859] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:52:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:52:16.065310 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b495c4-3582-4410-870d-5f2906e0867a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:16.066880 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d6b495c4-3582-4410-870d-5f2906e0867a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:16.079106 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b495c4-3582-4410-870d-5f2906e0867a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:16.084647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed3943b-e517-4318-8b4b-309344ffcf15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:16.085534 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3943b-e517-4318-8b4b-309344ffcf15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:16.085812 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3943b-e517-4318-8b4b-309344ffcf15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:16.086084 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3943b-e517-4318-8b4b-309344ffcf15 admin admin] PATH_INFO: `/v3/projects/06f62aa93e31427db87c39bcddee88b6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:16.087329 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed3943b-e517-4318-8b4b-309344ffcf15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06f62aa93e31427db87c39bcddee88b6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:16.089742 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed3943b-e517-4318-8b4b-309344ffcf15 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:16.139029 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fed3943b-e517-4318-8b4b-309344ffcf15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06f62aa93e31427db87c39bcddee88b6', 'request_id': 'req-fed3943b-e517-4318-8b4b-309344ffcf15'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:16.145698 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fed3943b-e517-4318-8b4b-309344ffcf15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06f62aa93e31427db87c39bcddee88b6', 'request_id': 'req-fed3943b-e517-4318-8b4b-309344ffcf15'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:16.152195 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fed3943b-e517-4318-8b4b-309344ffcf15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06f62aa93e31427db87c39bcddee88b6', 'request_id': 'req-fed3943b-e517-4318-8b4b-309344ffcf15'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:16.167477 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fed3943b-e517-4318-8b4b-309344ffcf15 admin admin] The token cache is being invalidate because project 06f62aa93e31427db87c39bcddee88b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:52:16.167760 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fed3943b-e517-4318-8b4b-309344ffcf15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:16.168395 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fed3943b-e517-4318-8b4b-309344ffcf15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:16.170188 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1450/2860] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:52:16 2022] DELETE /identity/v3/projects/06f62aa93e31427db87c39bcddee88b6 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:52:16.174194 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7468c2-e171-4832-89c6-c74056f80e52 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:16.184675 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7468c2-e171-4832-89c6-c74056f80e52 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:16.243327 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016ea66e-16dc-459e-b3ee-3d3659faaa06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:16.244467 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ea66e-16dc-459e-b3ee-3d3659faaa06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:16.244730 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ea66e-16dc-459e-b3ee-3d3659faaa06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:16.244990 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ea66e-16dc-459e-b3ee-3d3659faaa06 admin admin] PATH_INFO: `/v3/users/6166167ecb7049388db0349a19456281` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:16.246297 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ea66e-16dc-459e-b3ee-3d3659faaa06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6166167ecb7049388db0349a19456281)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:16.248236 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ea66e-16dc-459e-b3ee-3d3659faaa06 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:16.305980 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-016ea66e-16dc-459e-b3ee-3d3659faaa06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6166167ecb7049388db0349a19456281'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:16.307005 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-016ea66e-16dc-459e-b3ee-3d3659faaa06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6166167ecb7049388db0349a19456281'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:16.326781 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-016ea66e-16dc-459e-b3ee-3d3659faaa06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6166167ecb7049388db0349a19456281'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:16.335751 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-016ea66e-16dc-459e-b3ee-3d3659faaa06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6166167ecb7049388db0349a19456281'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:16.346779 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-016ea66e-16dc-459e-b3ee-3d3659faaa06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6166167ecb7049388db0349a19456281'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:16.355816 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1411/2861] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:52:16 2022] DELETE /identity/v3/users/6166167ecb7049388db0349a19456281 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:52:16.654642 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed3943b-e517-4318-8b4b-309344ffcf15 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:16.656219 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fed3943b-e517-4318-8b4b-309344ffcf15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:16.671450 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed3943b-e517-4318-8b4b-309344ffcf15 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:16.732989 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b1d9c6-aaff-434c-b032-635f101c0914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:16.733870 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1d9c6-aaff-434c-b032-635f101c0914 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:16.734143 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1d9c6-aaff-434c-b032-635f101c0914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:16.734459 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1d9c6-aaff-434c-b032-635f101c0914 admin admin] PATH_INFO: `/v3/projects/897b681b309d4f5c998c1ce57ebbbae7` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:16.735691 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b1d9c6-aaff-434c-b032-635f101c0914 admin admin] RBAC: Authorizing `identity:delete_project(project_id=897b681b309d4f5c998c1ce57ebbbae7)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:16.737620 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b1d9c6-aaff-434c-b032-635f101c0914 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:16.787192 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-02b1d9c6-aaff-434c-b032-635f101c0914 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '897b681b309d4f5c998c1ce57ebbbae7', 'request_id': 'req-02b1d9c6-aaff-434c-b032-635f101c0914'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:16.793400 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-02b1d9c6-aaff-434c-b032-635f101c0914 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '897b681b309d4f5c998c1ce57ebbbae7', 'request_id': 'req-02b1d9c6-aaff-434c-b032-635f101c0914'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:16.799620 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-02b1d9c6-aaff-434c-b032-635f101c0914 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '897b681b309d4f5c998c1ce57ebbbae7', 'request_id': 'req-02b1d9c6-aaff-434c-b032-635f101c0914'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:16.809150 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:52:16.814907 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-02b1d9c6-aaff-434c-b032-635f101c0914 admin admin] The token cache is being invalidate because project 897b681b309d4f5c998c1ce57ebbbae7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:52:16.815362 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-02b1d9c6-aaff-434c-b032-635f101c0914 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:16.816100 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-02b1d9c6-aaff-434c-b032-635f101c0914 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:16.818147 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1451/2862] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:52:16 2022] DELETE /identity/v3/projects/897b681b309d4f5c998c1ce57ebbbae7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:52:16.824073 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b363e-8da5-405c-ab1e-463a1c48af97 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:16.824627 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b363e-8da5-405c-ab1e-463a1c48af97 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:16.825009 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b363e-8da5-405c-ab1e-463a1c48af97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:16.851337 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-1a9b363e-8da5-405c-ab1e-463a1c48af97 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:52:16.852563 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1a9b363e-8da5-405c-ab1e-463a1c48af97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:16.885737 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:52:16.919239 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1412/2863] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:52:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:52:16.928470 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47279e0-6c2a-46b4-b66a-aa2b32fb2e6c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:16.928778 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47279e0-6c2a-46b4-b66a-aa2b32fb2e6c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:16.929029 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47279e0-6c2a-46b4-b66a-aa2b32fb2e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:16.951691 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b47279e0-6c2a-46b4-b66a-aa2b32fb2e6c None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:52:16.953058 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b47279e0-6c2a-46b4-b66a-aa2b32fb2e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:16.964264 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1452/2864] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:52:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:52:16.968819 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b363e-8da5-405c-ab1e-463a1c48af97 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:16.979902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b363e-8da5-405c-ab1e-463a1c48af97 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:16.985608 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacfdabd-70af-490f-b2b0-205f376eed36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:16.986324 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacfdabd-70af-490f-b2b0-205f376eed36 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:16.986612 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacfdabd-70af-490f-b2b0-205f376eed36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:16.986892 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacfdabd-70af-490f-b2b0-205f376eed36 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:16.987632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacfdabd-70af-490f-b2b0-205f376eed36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:52:16.987939 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacfdabd-70af-490f-b2b0-205f376eed36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:16.989565 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacfdabd-70af-490f-b2b0-205f376eed36 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:17.002852 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1413/2865] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:52:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:52:17.007546 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47279e0-6c2a-46b4-b66a-aa2b32fb2e6c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:17.017703 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47279e0-6c2a-46b4-b66a-aa2b32fb2e6c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:17.023159 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c67b78-dc5d-4c39-9f6f-69dedadbf5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:17.023826 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c67b78-dc5d-4c39-9f6f-69dedadbf5b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:17.024107 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c67b78-dc5d-4c39-9f6f-69dedadbf5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:17.024382 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c67b78-dc5d-4c39-9f6f-69dedadbf5b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:17.025071 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c67b78-dc5d-4c39-9f6f-69dedadbf5b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:17.027624 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c67b78-dc5d-4c39-9f6f-69dedadbf5b0 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:17.057561 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1453/2866] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:52:17 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:52:17.061672 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacfdabd-70af-490f-b2b0-205f376eed36 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:17.073104 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacfdabd-70af-490f-b2b0-205f376eed36 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:17.140298 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa58a3-bede-445a-9b82-41d06e23513a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:17.140896 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa58a3-bede-445a-9b82-41d06e23513a admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:17.141181 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa58a3-bede-445a-9b82-41d06e23513a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:17.141457 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa58a3-bede-445a-9b82-41d06e23513a admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:17.142151 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfa58a3-bede-445a-9b82-41d06e23513a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:17.144229 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfa58a3-bede-445a-9b82-41d06e23513a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:17.167988 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1414/2867] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:52:17 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:52:17.172333 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c67b78-dc5d-4c39-9f6f-69dedadbf5b0 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:17.183182 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c67b78-dc5d-4c39-9f6f-69dedadbf5b0 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:17.188507 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def583c3-0015-49a8-b358-0ce7201f81d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:17.189174 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def583c3-0015-49a8-b358-0ce7201f81d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:17.189466 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def583c3-0015-49a8-b358-0ce7201f81d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:17.189742 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def583c3-0015-49a8-b358-0ce7201f81d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:17.190515 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def583c3-0015-49a8-b358-0ce7201f81d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:17.192138 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def583c3-0015-49a8-b358-0ce7201f81d5 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:17.203349 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1454/2868] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:52:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:52:17.209572 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa58a3-bede-445a-9b82-41d06e23513a admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:17.220627 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa58a3-bede-445a-9b82-41d06e23513a admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:17.225826 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9539fac8-a932-4ae0-a845-6dff4b48979b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:17.226534 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9539fac8-a932-4ae0-a845-6dff4b48979b admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:17.226822 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9539fac8-a932-4ae0-a845-6dff4b48979b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:17.227098 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9539fac8-a932-4ae0-a845-6dff4b48979b admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:17.227838 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9539fac8-a932-4ae0-a845-6dff4b48979b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:17.229483 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9539fac8-a932-4ae0-a845-6dff4b48979b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:17.239526 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1415/2869] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:52:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 21:52:17.243743 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def583c3-0015-49a8-b358-0ce7201f81d5 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:17.253983 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def583c3-0015-49a8-b358-0ce7201f81d5 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:17.259371 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5948a-e7ec-4f38-94a0-6f1253f65742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:17.260312 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5948a-e7ec-4f38-94a0-6f1253f65742 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:17.260624 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5948a-e7ec-4f38-94a0-6f1253f65742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:17.260870 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5948a-e7ec-4f38-94a0-6f1253f65742 admin admin] PATH_INFO: `/v3/projects/f3da0d8fcea546d585c570f51cfce00d/users/542df236a0584e2ab1604181cae20d90/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:17.276095 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f5948a-e7ec-4f38-94a0-6f1253f65742 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3da0d8fcea546d585c570f51cfce00d, user_id=542df236a0584e2ab1604181cae20d90, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:17.278850 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f5948a-e7ec-4f38-94a0-6f1253f65742 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:17.293955 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1455/2870] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:52:17 2022] PUT /identity/v3/projects/f3da0d8fcea546d585c570f51cfce00d/users/542df236a0584e2ab1604181cae20d90/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:52:17.304576 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a8bfc-0e06-4a48-8b7c-30498cd1757d None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:17.304887 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a8bfc-0e06-4a48-8b7c-30498cd1757d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:17.305162 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a8bfc-0e06-4a48-8b7c-30498cd1757d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:17.342837 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-834a8bfc-0e06-4a48-8b7c-30498cd1757d None None] MFA Rules not processed for user `542df236a0584e2ab1604181cae20d90`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:52:17.344065 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-834a8bfc-0e06-4a48-8b7c-30498cd1757d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:17.419775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1416/2871] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:52:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:52:17.433444 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5948a-e7ec-4f38-94a0-6f1253f65742 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:17.436280 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-01f5948a-e7ec-4f38-94a0-6f1253f65742 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:17.514458 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5948a-e7ec-4f38-94a0-6f1253f65742 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:17.518609 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b02bc-81c9-4fb5-955d-45374c77e461 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:17.519260 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b02bc-81c9-4fb5-955d-45374c77e461 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:17.519521 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b02bc-81c9-4fb5-955d-45374c77e461 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:17.519803 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b02bc-81c9-4fb5-955d-45374c77e461 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:17.529257 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b02bc-81c9-4fb5-955d-45374c77e461 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:17.530732 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b02bc-81c9-4fb5-955d-45374c77e461 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:17.539873 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1456/2872] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:52:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:52:17.561023 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a8bfc-0e06-4a48-8b7c-30498cd1757d None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:17.595228 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:52:17.636315 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a8bfc-0e06-4a48-8b7c-30498cd1757d None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:17.642082 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d1cc5-94d3-43fd-94b5-2d3bfaa7ce30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:17.643559 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d1cc5-94d3-43fd-94b5-2d3bfaa7ce30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:17.643851 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d1cc5-94d3-43fd-94b5-2d3bfaa7ce30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:17.644109 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d1cc5-94d3-43fd-94b5-2d3bfaa7ce30 admin admin] PATH_INFO: `/v3/users/542df236a0584e2ab1604181cae20d90` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:17.645383 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7d1cc5-94d3-43fd-94b5-2d3bfaa7ce30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=542df236a0584e2ab1604181cae20d90)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:17.647377 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7d1cc5-94d3-43fd-94b5-2d3bfaa7ce30 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:17.725162 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-fd7d1cc5-94d3-43fd-94b5-2d3bfaa7ce30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '542df236a0584e2ab1604181cae20d90'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:17.726099 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-fd7d1cc5-94d3-43fd-94b5-2d3bfaa7ce30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '542df236a0584e2ab1604181cae20d90'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:17.739761 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-fd7d1cc5-94d3-43fd-94b5-2d3bfaa7ce30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '542df236a0584e2ab1604181cae20d90'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:17.747799 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-fd7d1cc5-94d3-43fd-94b5-2d3bfaa7ce30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '542df236a0584e2ab1604181cae20d90'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:17.757821 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-fd7d1cc5-94d3-43fd-94b5-2d3bfaa7ce30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '542df236a0584e2ab1604181cae20d90'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:17.764832 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1417/2873] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:52:17 2022] DELETE /identity/v3/users/542df236a0584e2ab1604181cae20d90 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:52:17.770357 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4ac2c-bbf3-429e-aa7b-8ac0d1a50f02 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:17.771045 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4ac2c-bbf3-429e-aa7b-8ac0d1a50f02 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:17.771156 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4ac2c-bbf3-429e-aa7b-8ac0d1a50f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:17.791185 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-55d4ac2c-bbf3-429e-aa7b-8ac0d1a50f02 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:52:17.792546 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-55d4ac2c-bbf3-429e-aa7b-8ac0d1a50f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:17.860158 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1457/2874] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:52:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:52:17.870064 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d1cc5-94d3-43fd-94b5-2d3bfaa7ce30 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:17.871638 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-fd7d1cc5-94d3-43fd-94b5-2d3bfaa7ce30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:17.946935 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d1cc5-94d3-43fd-94b5-2d3bfaa7ce30 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:17.952363 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b29df-dd1b-465b-bdc3-4e7abdcf94db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:17.953057 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b29df-dd1b-465b-bdc3-4e7abdcf94db service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:17.953344 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b29df-dd1b-465b-bdc3-4e7abdcf94db service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:17.953619 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b29df-dd1b-465b-bdc3-4e7abdcf94db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:17.965072 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b29df-dd1b-465b-bdc3-4e7abdcf94db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:17.966578 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b29df-dd1b-465b-bdc3-4e7abdcf94db service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:17.976959 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1418/2875] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:52:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:52:18.271808 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4ac2c-bbf3-429e-aa7b-8ac0d1a50f02 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:18.275173 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-55d4ac2c-bbf3-429e-aa7b-8ac0d1a50f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:18.292918 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4ac2c-bbf3-429e-aa7b-8ac0d1a50f02 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:18.299481 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21fef58-1e5f-4447-88fa-b0986a4ea206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:18.300586 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21fef58-1e5f-4447-88fa-b0986a4ea206 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:18.301009 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21fef58-1e5f-4447-88fa-b0986a4ea206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:18.301114 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21fef58-1e5f-4447-88fa-b0986a4ea206 admin admin] PATH_INFO: `/v3/projects/f3da0d8fcea546d585c570f51cfce00d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:18.302432 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21fef58-1e5f-4447-88fa-b0986a4ea206 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3da0d8fcea546d585c570f51cfce00d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:18.304361 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21fef58-1e5f-4447-88fa-b0986a4ea206 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:18.343794 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:52:18.358031 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d21fef58-1e5f-4447-88fa-b0986a4ea206 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3da0d8fcea546d585c570f51cfce00d', 'request_id': 'req-d21fef58-1e5f-4447-88fa-b0986a4ea206'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:18.364273 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d21fef58-1e5f-4447-88fa-b0986a4ea206 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3da0d8fcea546d585c570f51cfce00d', 'request_id': 'req-d21fef58-1e5f-4447-88fa-b0986a4ea206'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:18.370446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d21fef58-1e5f-4447-88fa-b0986a4ea206 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3da0d8fcea546d585c570f51cfce00d', 'request_id': 'req-d21fef58-1e5f-4447-88fa-b0986a4ea206'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:18.385234 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d21fef58-1e5f-4447-88fa-b0986a4ea206 admin admin] The token cache is being invalidate because project f3da0d8fcea546d585c570f51cfce00d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:52:18.385407 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d21fef58-1e5f-4447-88fa-b0986a4ea206 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:18.386013 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d21fef58-1e5f-4447-88fa-b0986a4ea206 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:18.387620 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1458/2876] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:52:18 2022] DELETE /identity/v3/projects/f3da0d8fcea546d585c570f51cfce00d => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:52:18.392617 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3facfd10-97b9-4b3d-a56b-07ed22f995ff None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:18.392898 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3facfd10-97b9-4b3d-a56b-07ed22f995ff None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:18.393174 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3facfd10-97b9-4b3d-a56b-07ed22f995ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:18.412525 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-3facfd10-97b9-4b3d-a56b-07ed22f995ff None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:52:18.413729 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3facfd10-97b9-4b3d-a56b-07ed22f995ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:18.476889 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:52:18.498653 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1419/2877] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:52:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:52:18.509537 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efa899-5c6b-42d1-b2e1-c29fc95f6c8a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:18.509994 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efa899-5c6b-42d1-b2e1-c29fc95f6c8a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:18.510115 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efa899-5c6b-42d1-b2e1-c29fc95f6c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:18.530104 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-79efa899-5c6b-42d1-b2e1-c29fc95f6c8a None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:52:18.531280 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-79efa899-5c6b-42d1-b2e1-c29fc95f6c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:18.540026 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1459/2878] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:52:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:52:18.544600 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3facfd10-97b9-4b3d-a56b-07ed22f995ff None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:18.555339 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3facfd10-97b9-4b3d-a56b-07ed22f995ff None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:18.561217 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11acb333-7420-4e61-885d-19f1812b1bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:18.561889 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11acb333-7420-4e61-885d-19f1812b1bb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:18.562153 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11acb333-7420-4e61-885d-19f1812b1bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:18.562455 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11acb333-7420-4e61-885d-19f1812b1bb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:18.563251 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11acb333-7420-4e61-885d-19f1812b1bb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:52:18.563562 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11acb333-7420-4e61-885d-19f1812b1bb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:18.565269 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11acb333-7420-4e61-885d-19f1812b1bb2 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:18.577725 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1420/2879] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:52:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:52:18.582625 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79efa899-5c6b-42d1-b2e1-c29fc95f6c8a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:18.594912 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79efa899-5c6b-42d1-b2e1-c29fc95f6c8a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:18.602647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256b3234-3402-441b-ab76-6c7e7e4b98da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:18.603599 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b3234-3402-441b-ab76-6c7e7e4b98da admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:18.603883 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b3234-3402-441b-ab76-6c7e7e4b98da admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:18.604156 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b3234-3402-441b-ab76-6c7e7e4b98da admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:18.604850 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256b3234-3402-441b-ab76-6c7e7e4b98da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:18.606796 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256b3234-3402-441b-ab76-6c7e7e4b98da admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:18.633984 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1460/2880] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:52:18 2022] POST /identity/v3/projects => generated 391 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:52:18.639990 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11acb333-7420-4e61-885d-19f1812b1bb2 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:18.651758 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11acb333-7420-4e61-885d-19f1812b1bb2 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:18.724654 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a70c79e-c5b1-493f-a626-b14f547da6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:18.725328 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70c79e-c5b1-493f-a626-b14f547da6be admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:18.725607 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70c79e-c5b1-493f-a626-b14f547da6be admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:18.725906 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70c79e-c5b1-493f-a626-b14f547da6be admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:18.726634 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a70c79e-c5b1-493f-a626-b14f547da6be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:18.728634 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a70c79e-c5b1-493f-a626-b14f547da6be admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:18.751021 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1421/2881] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:52:18 2022] POST /identity/v3/users => generated 296 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:52:18.755486 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256b3234-3402-441b-ab76-6c7e7e4b98da admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:18.766089 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256b3234-3402-441b-ab76-6c7e7e4b98da admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:18.771282 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f63b4-66eb-4b38-b0b6-b5ff3ecb6390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:18.771952 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f63b4-66eb-4b38-b0b6-b5ff3ecb6390 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:18.772232 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f63b4-66eb-4b38-b0b6-b5ff3ecb6390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:18.772517 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f63b4-66eb-4b38-b0b6-b5ff3ecb6390 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:18.773248 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7f63b4-66eb-4b38-b0b6-b5ff3ecb6390 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:18.774869 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7f63b4-66eb-4b38-b0b6-b5ff3ecb6390 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:18.785504 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1461/2882] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:52:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 21:52:18.790085 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a70c79e-c5b1-493f-a626-b14f547da6be admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:18.801875 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a70c79e-c5b1-493f-a626-b14f547da6be admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:18.808030 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc5f172-706e-4fab-9b66-8df62a0ac07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:18.808708 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5f172-706e-4fab-9b66-8df62a0ac07b admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:18.809028 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5f172-706e-4fab-9b66-8df62a0ac07b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:18.809307 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5f172-706e-4fab-9b66-8df62a0ac07b admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:18.810040 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc5f172-706e-4fab-9b66-8df62a0ac07b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:18.811682 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc5f172-706e-4fab-9b66-8df62a0ac07b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:18.822226 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1422/2883] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:52:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:52:18.827906 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f63b4-66eb-4b38-b0b6-b5ff3ecb6390 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:18.844471 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f63b4-66eb-4b38-b0b6-b5ff3ecb6390 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:18.853271 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0025ec-4e13-4334-932d-8d9638d3147b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:18.854743 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0025ec-4e13-4334-932d-8d9638d3147b admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:18.855027 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0025ec-4e13-4334-932d-8d9638d3147b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:18.855286 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0025ec-4e13-4334-932d-8d9638d3147b admin admin] PATH_INFO: `/v3/projects/3b3b56b4f6014c62a7ef0b81810bb009/users/1017a887724f4c2e8fef8cfd277771cd/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:18.871887 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0025ec-4e13-4334-932d-8d9638d3147b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b3b56b4f6014c62a7ef0b81810bb009, user_id=1017a887724f4c2e8fef8cfd277771cd, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:18.874716 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0025ec-4e13-4334-932d-8d9638d3147b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:18.889831 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1462/2884] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:52:18 2022] PUT /identity/v3/projects/3b3b56b4f6014c62a7ef0b81810bb009/users/1017a887724f4c2e8fef8cfd277771cd/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:52:18.953937 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339c832-f3c4-4b44-8d6a-2613a6bf84f9 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:18.954561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339c832-f3c4-4b44-8d6a-2613a6bf84f9 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:18.954561 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339c832-f3c4-4b44-8d6a-2613a6bf84f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:18.987477 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-a339c832-f3c4-4b44-8d6a-2613a6bf84f9 None None] MFA Rules not processed for user `1017a887724f4c2e8fef8cfd277771cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:52:18.988697 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a339c832-f3c4-4b44-8d6a-2613a6bf84f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:19.067673 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1423/2885] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:52:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:52:19.079533 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0025ec-4e13-4334-932d-8d9638d3147b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:19.081941 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cf0025ec-4e13-4334-932d-8d9638d3147b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:19.153044 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0025ec-4e13-4334-932d-8d9638d3147b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:19.158103 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eae862-fc4e-46ad-9fd9-80dc1372af5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:19.158801 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eae862-fc4e-46ad-9fd9-80dc1372af5f service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:19.159086 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eae862-fc4e-46ad-9fd9-80dc1372af5f service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:19.159361 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eae862-fc4e-46ad-9fd9-80dc1372af5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:19.169312 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eae862-fc4e-46ad-9fd9-80dc1372af5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:19.170806 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eae862-fc4e-46ad-9fd9-80dc1372af5f service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:19.180356 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1463/2886] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:52:19 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:52:29.064697 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=76,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:52:29.175763 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=65,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:52:40.650460 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339c832-f3c4-4b44-8d6a-2613a6bf84f9 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:40.729717 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339c832-f3c4-4b44-8d6a-2613a6bf84f9 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:40.737130 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623de053-1959-41ec-a085-f194672c1e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:40.738035 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623de053-1959-41ec-a085-f194672c1e33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:40.738354 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623de053-1959-41ec-a085-f194672c1e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:40.738639 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623de053-1959-41ec-a085-f194672c1e33 admin admin] PATH_INFO: `/v3/users/1017a887724f4c2e8fef8cfd277771cd` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:40.740040 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623de053-1959-41ec-a085-f194672c1e33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1017a887724f4c2e8fef8cfd277771cd)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:40.741993 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623de053-1959-41ec-a085-f194672c1e33 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:40.793626 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-623de053-1959-41ec-a085-f194672c1e33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1017a887724f4c2e8fef8cfd277771cd'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:40.794310 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-623de053-1959-41ec-a085-f194672c1e33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1017a887724f4c2e8fef8cfd277771cd'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:40.805878 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-623de053-1959-41ec-a085-f194672c1e33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1017a887724f4c2e8fef8cfd277771cd'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:40.813448 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-623de053-1959-41ec-a085-f194672c1e33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1017a887724f4c2e8fef8cfd277771cd'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:40.822495 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-623de053-1959-41ec-a085-f194672c1e33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1017a887724f4c2e8fef8cfd277771cd'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:40.829967 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1424/2887] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:52:40 2022] DELETE /identity/v3/users/1017a887724f4c2e8fef8cfd277771cd => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:52:40.836090 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9ec3d-6f73-49bf-9ce1-0119f9dc9304 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:40.836633 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9ec3d-6f73-49bf-9ce1-0119f9dc9304 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:40.836758 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9ec3d-6f73-49bf-9ce1-0119f9dc9304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:40.857911 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-8db9ec3d-6f73-49bf-9ce1-0119f9dc9304 None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:52:40.859109 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8db9ec3d-6f73-49bf-9ce1-0119f9dc9304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:40.930079 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1464/2888] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:52:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:52:40.940104 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623de053-1959-41ec-a085-f194672c1e33 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:40.941677 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-623de053-1959-41ec-a085-f194672c1e33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:41.006505 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623de053-1959-41ec-a085-f194672c1e33 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:41.010713 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa9462d-c6ff-4b55-940d-dd4aab460231 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:41.011368 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9462d-c6ff-4b55-940d-dd4aab460231 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:41.011646 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9462d-c6ff-4b55-940d-dd4aab460231 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:41.011918 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9462d-c6ff-4b55-940d-dd4aab460231 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:41.022267 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa9462d-c6ff-4b55-940d-dd4aab460231 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:41.023869 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa9462d-c6ff-4b55-940d-dd4aab460231 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:41.033652 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1425/2889] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:52:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:52:41.320357 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db9ec3d-6f73-49bf-9ce1-0119f9dc9304 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:41.321914 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8db9ec3d-6f73-49bf-9ce1-0119f9dc9304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:41.335099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db9ec3d-6f73-49bf-9ce1-0119f9dc9304 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:41.340062 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650bf9f4-e706-4777-bec2-9c8d58c649ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:41.341259 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650bf9f4-e706-4777-bec2-9c8d58c649ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:41.341569 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650bf9f4-e706-4777-bec2-9c8d58c649ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:41.341847 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650bf9f4-e706-4777-bec2-9c8d58c649ff admin admin] PATH_INFO: `/v3/projects/3b3b56b4f6014c62a7ef0b81810bb009` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:41.343087 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650bf9f4-e706-4777-bec2-9c8d58c649ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b3b56b4f6014c62a7ef0b81810bb009)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:41.345005 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650bf9f4-e706-4777-bec2-9c8d58c649ff admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:41.398004 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-650bf9f4-e706-4777-bec2-9c8d58c649ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b3b56b4f6014c62a7ef0b81810bb009', 'request_id': 'req-650bf9f4-e706-4777-bec2-9c8d58c649ff'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:41.404706 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-650bf9f4-e706-4777-bec2-9c8d58c649ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b3b56b4f6014c62a7ef0b81810bb009', 'request_id': 'req-650bf9f4-e706-4777-bec2-9c8d58c649ff'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:41.410760 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-650bf9f4-e706-4777-bec2-9c8d58c649ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b3b56b4f6014c62a7ef0b81810bb009', 'request_id': 'req-650bf9f4-e706-4777-bec2-9c8d58c649ff'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:41.425621 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-650bf9f4-e706-4777-bec2-9c8d58c649ff admin admin] The token cache is being invalidate because project 3b3b56b4f6014c62a7ef0b81810bb009 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:52:41.425935 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-650bf9f4-e706-4777-bec2-9c8d58c649ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:41.426662 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-650bf9f4-e706-4777-bec2-9c8d58c649ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:52:41.428171 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1465/2890] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:52:41 2022] DELETE /identity/v3/projects/3b3b56b4f6014c62a7ef0b81810bb009 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:52:41.434636 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d287e2-9b1f-4f98-bab5-87c1980db64f None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:41.435103 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d287e2-9b1f-4f98-bab5-87c1980db64f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:41.435385 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d287e2-9b1f-4f98-bab5-87c1980db64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:41.458865 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-77d287e2-9b1f-4f98-bab5-87c1980db64f None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:52:41.460072 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-77d287e2-9b1f-4f98-bab5-87c1980db64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:41.479539 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:52:41.527754 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1426/2891] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:52:41.539977 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2032ff-c3fd-4aea-ab7f-d09c662020fa None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:41.540508 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2032ff-c3fd-4aea-ab7f-d09c662020fa None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:41.540940 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2032ff-c3fd-4aea-ab7f-d09c662020fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:41.565256 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-ac2032ff-c3fd-4aea-ab7f-d09c662020fa None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:52:41.566506 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ac2032ff-c3fd-4aea-ab7f-d09c662020fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:41.576574 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1466/2892] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:52:41.581513 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d287e2-9b1f-4f98-bab5-87c1980db64f None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:41.594232 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d287e2-9b1f-4f98-bab5-87c1980db64f None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:41.601626 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a005cd3-9180-405d-b818-c20adcb10737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:41.602647 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a005cd3-9180-405d-b818-c20adcb10737 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:41.603065 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a005cd3-9180-405d-b818-c20adcb10737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:41.603484 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a005cd3-9180-405d-b818-c20adcb10737 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:41.604624 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a005cd3-9180-405d-b818-c20adcb10737 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 21:52:41.605077 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a005cd3-9180-405d-b818-c20adcb10737 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:41.607356 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a005cd3-9180-405d-b818-c20adcb10737 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:41.625512 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1427/2893] 10.222.0.65 () {58 vars in 1152 bytes} [Thu May 19 21:52:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 21:52:41.630980 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2032ff-c3fd-4aea-ab7f-d09c662020fa None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:41.642619 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2032ff-c3fd-4aea-ab7f-d09c662020fa None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:41.651587 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb65ac0c-bc26-45c8-b618-303c8d393bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:41.652358 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb65ac0c-bc26-45c8-b618-303c8d393bb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:41.652657 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb65ac0c-bc26-45c8-b618-303c8d393bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:41.653101 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb65ac0c-bc26-45c8-b618-303c8d393bb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:41.654025 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb65ac0c-bc26-45c8-b618-303c8d393bb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:41.656017 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb65ac0c-bc26-45c8-b618-303c8d393bb4 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:41.687922 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1467/2894] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:52:41 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:52:41.692415 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a005cd3-9180-405d-b818-c20adcb10737 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:41.704383 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a005cd3-9180-405d-b818-c20adcb10737 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:41.766652 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2c9b03-929c-494d-84f8-5d86b320b5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:41.767354 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2c9b03-929c-494d-84f8-5d86b320b5f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:41.767669 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2c9b03-929c-494d-84f8-5d86b320b5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:41.767962 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2c9b03-929c-494d-84f8-5d86b320b5f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:41.768652 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2c9b03-929c-494d-84f8-5d86b320b5f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:41.770632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2c9b03-929c-494d-84f8-5d86b320b5f7 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:41.796496 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1428/2895] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:52:41 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:52:41.804841 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb65ac0c-bc26-45c8-b618-303c8d393bb4 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:41.815815 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb65ac0c-bc26-45c8-b618-303c8d393bb4 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:41.820911 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e41e3-b13f-421d-bd45-c4fa04cbc55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:41.821579 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e41e3-b13f-421d-bd45-c4fa04cbc55e admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:41.821865 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e41e3-b13f-421d-bd45-c4fa04cbc55e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:41.822141 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e41e3-b13f-421d-bd45-c4fa04cbc55e admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:41.822904 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e41e3-b13f-421d-bd45-c4fa04cbc55e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:41.824519 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e41e3-b13f-421d-bd45-c4fa04cbc55e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:41.836857 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1468/2896] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:52:41.841512 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2c9b03-929c-494d-84f8-5d86b320b5f7 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:41.853605 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2c9b03-929c-494d-84f8-5d86b320b5f7 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:41.859415 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08147196-aab3-4906-9556-278611cec4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:41.860104 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08147196-aab3-4906-9556-278611cec4f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:41.860395 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08147196-aab3-4906-9556-278611cec4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:41.860664 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08147196-aab3-4906-9556-278611cec4f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:41.861407 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08147196-aab3-4906-9556-278611cec4f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:41.863103 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08147196-aab3-4906-9556-278611cec4f2 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:41.874117 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1429/2897] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:52:41.878585 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e41e3-b13f-421d-bd45-c4fa04cbc55e admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:41.888973 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e41e3-b13f-421d-bd45-c4fa04cbc55e admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:41.894554 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98600927-bffb-4491-af59-bf9cb093d31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:41.895454 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98600927-bffb-4491-af59-bf9cb093d31c admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:41.895734 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98600927-bffb-4491-af59-bf9cb093d31c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:41.896008 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98600927-bffb-4491-af59-bf9cb093d31c admin admin] PATH_INFO: `/v3/projects/52d49b95be55483f93b8a325d6bff693/users/f69162209070421faef53c4f75803f38/roles/70699fb0eae7441999835fa3b50350b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:41.912113 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98600927-bffb-4491-af59-bf9cb093d31c admin admin] RBAC: Authorizing `identity:create_grant(project_id=52d49b95be55483f93b8a325d6bff693, user_id=f69162209070421faef53c4f75803f38, role_id=70699fb0eae7441999835fa3b50350b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:41.914719 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98600927-bffb-4491-af59-bf9cb093d31c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:41.930544 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1469/2898] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:52:41 2022] PUT /identity/v3/projects/52d49b95be55483f93b8a325d6bff693/users/f69162209070421faef53c4f75803f38/roles/70699fb0eae7441999835fa3b50350b3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:52:41.935852 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a5cd58-6b98-43ee-97f7-b1c16261ba1e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:41.936131 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a5cd58-6b98-43ee-97f7-b1c16261ba1e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:41.936421 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a5cd58-6b98-43ee-97f7-b1c16261ba1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:41.956126 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-61a5cd58-6b98-43ee-97f7-b1c16261ba1e None None] MFA Rules not processed for user `d45bbb3805754099870efea6f5d53dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:52:41.957352 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-61a5cd58-6b98-43ee-97f7-b1c16261ba1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:42.025387 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1430/2899] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:52:42.039057 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98600927-bffb-4491-af59-bf9cb093d31c admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:42.041845 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-98600927-bffb-4491-af59-bf9cb093d31c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:42.068353 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:52:42.116465 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98600927-bffb-4491-af59-bf9cb093d31c admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:42.121016 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c519fc1-1a63-4275-940f-76e51577f302 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:42.121711 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c519fc1-1a63-4275-940f-76e51577f302 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:42.122069 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c519fc1-1a63-4275-940f-76e51577f302 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:42.122645 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c519fc1-1a63-4275-940f-76e51577f302 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:42.133010 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c519fc1-1a63-4275-940f-76e51577f302 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:42.134519 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c519fc1-1a63-4275-940f-76e51577f302 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:42.145621 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1470/2900] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:52:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:52:46.264297 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637db20-ee41-4940-9c4f-ec7d47b846ec None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:46.264877 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637db20-ee41-4940-9c4f-ec7d47b846ec None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:46.264877 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637db20-ee41-4940-9c4f-ec7d47b846ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:46.297747 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-5637db20-ee41-4940-9c4f-ec7d47b846ec None None] MFA Rules not processed for user `f69162209070421faef53c4f75803f38`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:52:46.299121 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5637db20-ee41-4940-9c4f-ec7d47b846ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:46.319215 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:52:46.378951 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1431/2901] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:52:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:52:46.383619 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c519fc1-1a63-4275-940f-76e51577f302 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:46.394202 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c519fc1-1a63-4275-940f-76e51577f302 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:46.399111 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4c326-84ed-49aa-b004-dad38fa8b60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:46.399779 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4c326-84ed-49aa-b004-dad38fa8b60a admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:46.400058 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4c326-84ed-49aa-b004-dad38fa8b60a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:46.400335 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4c326-84ed-49aa-b004-dad38fa8b60a admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:46.401030 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e4c326-84ed-49aa-b004-dad38fa8b60a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:46.402997 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e4c326-84ed-49aa-b004-dad38fa8b60a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:46.430693 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1471/2902] 10.222.0.65 () {60 vars in 1152 bytes} [Thu May 19 21:52:46 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:52:46.435346 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5637db20-ee41-4940-9c4f-ec7d47b846ec None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:46.447438 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5637db20-ee41-4940-9c4f-ec7d47b846ec None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:46.515837 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc1a877-1ba3-44d4-bf5d-1eff589ff36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:46.516523 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc1a877-1ba3-44d4-bf5d-1eff589ff36c admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:46.516808 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc1a877-1ba3-44d4-bf5d-1eff589ff36c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:46.517078 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc1a877-1ba3-44d4-bf5d-1eff589ff36c admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:46.517773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc1a877-1ba3-44d4-bf5d-1eff589ff36c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:46.519786 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc1a877-1ba3-44d4-bf5d-1eff589ff36c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:46.555624 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1432/2903] 10.222.0.65 () {60 vars in 1143 bytes} [Thu May 19 21:52:46 2022] POST /identity/v3/users => generated 292 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 21:52:46.561031 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4c326-84ed-49aa-b004-dad38fa8b60a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:46.572264 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4c326-84ed-49aa-b004-dad38fa8b60a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:46.577757 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2506896-b6d6-4088-8848-b882ca78f8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:46.578542 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2506896-b6d6-4088-8848-b882ca78f8ea admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:46.578826 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2506896-b6d6-4088-8848-b882ca78f8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:46.579100 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2506896-b6d6-4088-8848-b882ca78f8ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:46.579833 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2506896-b6d6-4088-8848-b882ca78f8ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:46.581858 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2506896-b6d6-4088-8848-b882ca78f8ea admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:46.594221 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1472/2904] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:52:46.598608 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc1a877-1ba3-44d4-bf5d-1eff589ff36c admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:46.611467 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc1a877-1ba3-44d4-bf5d-1eff589ff36c admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:46.616369 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2a4c8-db5a-4731-bd60-420a5a71ba76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:46.617283 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2a4c8-db5a-4731-bd60-420a5a71ba76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:46.617599 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2a4c8-db5a-4731-bd60-420a5a71ba76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:46.617904 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2a4c8-db5a-4731-bd60-420a5a71ba76 admin admin] PATH_INFO: `/v3/projects/6154d8bf3c2f4119b0bdd36923a0137d/users/b982c54ed4ed47dc8cf55a2c34001db8/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:46.634198 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda2a4c8-db5a-4731-bd60-420a5a71ba76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6154d8bf3c2f4119b0bdd36923a0137d, user_id=b982c54ed4ed47dc8cf55a2c34001db8, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:46.637572 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda2a4c8-db5a-4731-bd60-420a5a71ba76 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:46.654085 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1433/2905] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:52:46 2022] PUT /identity/v3/projects/6154d8bf3c2f4119b0bdd36923a0137d/users/b982c54ed4ed47dc8cf55a2c34001db8/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:52:46.658286 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2506896-b6d6-4088-8848-b882ca78f8ea admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:46.670527 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2506896-b6d6-4088-8848-b882ca78f8ea admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:46.735743 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e7bd70-8553-4a28-b545-4648365938be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:46.736448 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7bd70-8553-4a28-b545-4648365938be admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:46.736725 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7bd70-8553-4a28-b545-4648365938be admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:46.736997 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7bd70-8553-4a28-b545-4648365938be admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:46.737770 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e7bd70-8553-4a28-b545-4648365938be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:46.739592 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e7bd70-8553-4a28-b545-4648365938be admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:46.754428 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1473/2906] 10.222.0.65 () {58 vars in 1121 bytes} [Thu May 19 21:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 21:52:46.755850 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2a4c8-db5a-4731-bd60-420a5a71ba76 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:46.767437 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2a4c8-db5a-4731-bd60-420a5a71ba76 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:46.772508 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad2e2b0-dd7d-475d-95b0-56ebdfba6d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:46.773503 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2e2b0-dd7d-475d-95b0-56ebdfba6d2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:46.773876 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2e2b0-dd7d-475d-95b0-56ebdfba6d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:46.774218 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2e2b0-dd7d-475d-95b0-56ebdfba6d2a admin admin] PATH_INFO: `/v3/projects/6154d8bf3c2f4119b0bdd36923a0137d/users/b982c54ed4ed47dc8cf55a2c34001db8/roles/2b714f2e5c6c469f8fa037bda1ed16f6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:46.776382 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad2e2b0-dd7d-475d-95b0-56ebdfba6d2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6154d8bf3c2f4119b0bdd36923a0137d, user_id=b982c54ed4ed47dc8cf55a2c34001db8, role_id=2b714f2e5c6c469f8fa037bda1ed16f6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:46.779277 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad2e2b0-dd7d-475d-95b0-56ebdfba6d2a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:46.790619 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1434/2907] 10.222.0.65 () {60 vars in 1482 bytes} [Thu May 19 21:52:46 2022] PUT /identity/v3/projects/6154d8bf3c2f4119b0bdd36923a0137d/users/b982c54ed4ed47dc8cf55a2c34001db8/roles/2b714f2e5c6c469f8fa037bda1ed16f6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:52:51.278784 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3dd6a-f76b-428d-b77d-c6c15142d652 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:51.279304 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3dd6a-f76b-428d-b77d-c6c15142d652 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:51.279429 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3dd6a-f76b-428d-b77d-c6c15142d652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:51.307467 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-45c3dd6a-f76b-428d-b77d-c6c15142d652 None None] MFA Rules not processed for user `b982c54ed4ed47dc8cf55a2c34001db8`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:52:51.308637 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-45c3dd6a-f76b-428d-b77d-c6c15142d652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:51.382570 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1474/2908] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:52:51 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:52:51.400924 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad2e2b0-dd7d-475d-95b0-56ebdfba6d2a admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:52:51.403744 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6ad2e2b0-dd7d-475d-95b0-56ebdfba6d2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:51.494708 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad2e2b0-dd7d-475d-95b0-56ebdfba6d2a admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:52:51.500690 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4550a9b-f611-4871-9ecf-aca9128fe8ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:52:51.501356 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4550a9b-f611-4871-9ecf-aca9128fe8ff service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:51.501635 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4550a9b-f611-4871-9ecf-aca9128fe8ff service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:51.501907 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4550a9b-f611-4871-9ecf-aca9128fe8ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:51.507511 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:52:51.568958 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4550a9b-f611-4871-9ecf-aca9128fe8ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:52:51.570694 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4550a9b-f611-4871-9ecf-aca9128fe8ff service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:52:51.581684 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1435/2909] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:52:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:52:52.624465 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86311bb-e546-46f9-85fd-78493c86256e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:52:52.624995 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86311bb-e546-46f9-85fd-78493c86256e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:52:52.624995 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86311bb-e546-46f9-85fd-78493c86256e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:52:52.643917 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f86311bb-e546-46f9-85fd-78493c86256e None None] MFA Rules not processed for user `f69162209070421faef53c4f75803f38`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:52:52.645235 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f86311bb-e546-46f9-85fd-78493c86256e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:52:52.652686 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1475/2910] 10.222.0.65 () {58 vars in 957 bytes} [Thu May 19 21:52:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:53:01.576931 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=24 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:53:02.638267 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:53:12.178557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4550a9b-f611-4871-9ecf-aca9128fe8ff service nova] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:53:12.180356 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c4550a9b-f611-4871-9ecf-aca9128fe8ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:12.240158 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4550a9b-f611-4871-9ecf-aca9128fe8ff service nova] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:53:12.244507 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e20cc9-39b4-4316-88d9-ffe444b05ae5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51852131d4e5448f8152a51159bd78e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:53:12.245314 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e20cc9-39b4-4316-88d9-ffe444b05ae5 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:12.245591 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e20cc9-39b4-4316-88d9-ffe444b05ae5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:12.245867 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e20cc9-39b4-4316-88d9-ffe444b05ae5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:12.247848 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-48e20cc9-39b4-4316-88d9-ffe444b05ae5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:12.262287 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e20cc9-39b4-4316-88d9-ffe444b05ae5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:53:12.263752 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e20cc9-39b4-4316-88d9-ffe444b05ae5 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:53:12.276902 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1436/2911] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:53:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:53:12.593894 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86311bb-e546-46f9-85fd-78493c86256e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:53:12.595568 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f86311bb-e546-46f9-85fd-78493c86256e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:12.679633 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86311bb-e546-46f9-85fd-78493c86256e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:53:12.684984 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f24c9-bccc-4cb8-8ae0-997d073145d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:53:12.685792 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f24c9-bccc-4cb8-8ae0-997d073145d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:12.686070 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f24c9-bccc-4cb8-8ae0-997d073145d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:12.686374 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f24c9-bccc-4cb8-8ae0-997d073145d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:12.688649 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1e9f24c9-bccc-4cb8-8ae0-997d073145d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:12.699713 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f24c9-bccc-4cb8-8ae0-997d073145d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:53:12.701154 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f24c9-bccc-4cb8-8ae0-997d073145d9 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:53:12.709971 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1476/2912] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:53:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:53:13.213060 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e20cc9-39b4-4316-88d9-ffe444b05ae5 service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:53:13.222748 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e20cc9-39b4-4316-88d9-ffe444b05ae5 service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:53:13.227115 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77804fa0-b48a-442d-9f44-5deaee040139 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:53:13.227763 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77804fa0-b48a-442d-9f44-5deaee040139 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:13.228042 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77804fa0-b48a-442d-9f44-5deaee040139 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:13.228314 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77804fa0-b48a-442d-9f44-5deaee040139 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:13.230206 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-77804fa0-b48a-442d-9f44-5deaee040139 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:13.244587 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77804fa0-b48a-442d-9f44-5deaee040139 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:53:13.246040 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77804fa0-b48a-442d-9f44-5deaee040139 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:53:13.258663 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1437/2913] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:53:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:53:14.978014 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f24c9-bccc-4cb8-8ae0-997d073145d9 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:53:14.988434 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f24c9-bccc-4cb8-8ae0-997d073145d9 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:53:14.992064 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c3602-1f45-4e29-8f4b-c8ef3f948298 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:53:14.992725 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c3602-1f45-4e29-8f4b-c8ef3f948298 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:14.993010 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c3602-1f45-4e29-8f4b-c8ef3f948298 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:14.993339 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c3602-1f45-4e29-8f4b-c8ef3f948298 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:14.995564 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f54c3602-1f45-4e29-8f4b-c8ef3f948298 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:53:15.006482 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c3602-1f45-4e29-8f4b-c8ef3f948298 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:53:15.008027 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c3602-1f45-4e29-8f4b-c8ef3f948298 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:53:15.017801 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1477/2914] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:53:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:53:23.252937 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=32 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:53:25.014343 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=32 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:53:27.991833 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77804fa0-b48a-442d-9f44-5deaee040139 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:53:28.001178 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77804fa0-b48a-442d-9f44-5deaee040139 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:53:28.005928 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a9ef7-a161-4c2c-895b-2208b907ae5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:53:28.006607 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a9ef7-a161-4c2c-895b-2208b907ae5a service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:53:28.006887 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a9ef7-a161-4c2c-895b-2208b907ae5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:53:28.007158 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a9ef7-a161-4c2c-895b-2208b907ae5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:53:28.017172 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a9ef7-a161-4c2c-895b-2208b907ae5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:53:28.018645 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a9ef7-a161-4c2c-895b-2208b907ae5a service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:53:28.028479 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1438/2915] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:53:27 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:53:38.023897 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=6 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:55:58.305957 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c3602-1f45-4e29-8f4b-c8ef3f948298 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:55:58.345811 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c3602-1f45-4e29-8f4b-c8ef3f948298 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:55:58.350595 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7fa2b3-cf59-4a43-9212-453a10f7de6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:55:58.351240 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7fa2b3-cf59-4a43-9212-453a10f7de6b service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:55:58.351521 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7fa2b3-cf59-4a43-9212-453a10f7de6b service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:55:58.351788 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7fa2b3-cf59-4a43-9212-453a10f7de6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:55:58.353883 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ae7fa2b3-cf59-4a43-9212-453a10f7de6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:55:58.385821 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7fa2b3-cf59-4a43-9212-453a10f7de6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:55:58.387320 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7fa2b3-cf59-4a43-9212-453a10f7de6b service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:55:58.396116 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1478/2916] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:55:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:55:58.502731 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a9ef7-a161-4c2c-895b-2208b907ae5a service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:55:58.535106 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a9ef7-a161-4c2c-895b-2208b907ae5a service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:55:58.539306 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef993147-a9e7-4d3d-96cd-4f8b81f375bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:55:58.540069 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef993147-a9e7-4d3d-96cd-4f8b81f375bd service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:55:58.540452 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef993147-a9e7-4d3d-96cd-4f8b81f375bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:55:58.540803 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef993147-a9e7-4d3d-96cd-4f8b81f375bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:55:58.542739 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ef993147-a9e7-4d3d-96cd-4f8b81f375bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:55:58.553518 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef993147-a9e7-4d3d-96cd-4f8b81f375bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:55:58.554999 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef993147-a9e7-4d3d-96cd-4f8b81f375bd service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:55:58.563150 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1439/2917] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:55:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:55:58.653252 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7fa2b3-cf59-4a43-9212-453a10f7de6b service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:55:58.662993 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7fa2b3-cf59-4a43-9212-453a10f7de6b service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:55:58.666714 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4a94e7-072f-4e0c-bb24-b8934ad33c4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:55:58.667360 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4a94e7-072f-4e0c-bb24-b8934ad33c4e service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:55:58.667901 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4a94e7-072f-4e0c-bb24-b8934ad33c4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:55:58.668182 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4a94e7-072f-4e0c-bb24-b8934ad33c4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:55:58.670144 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4d4a94e7-072f-4e0c-bb24-b8934ad33c4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:55:58.725906 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4a94e7-072f-4e0c-bb24-b8934ad33c4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:55:58.727303 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4a94e7-072f-4e0c-bb24-b8934ad33c4e service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:55:58.738496 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1479/2918] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:55:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:55:58.804127 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef993147-a9e7-4d3d-96cd-4f8b81f375bd service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:55:58.805784 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ef993147-a9e7-4d3d-96cd-4f8b81f375bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:55:58.879747 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef993147-a9e7-4d3d-96cd-4f8b81f375bd service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:55:58.885307 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47309f60-198a-40ad-bb5d-9cfc8b5bfd3f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6143bddc05a46ea972709576c7ad95f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:55:58.885961 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47309f60-198a-40ad-bb5d-9cfc8b5bfd3f service placement] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:55:58.886257 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47309f60-198a-40ad-bb5d-9cfc8b5bfd3f service placement] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:55:58.886552 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47309f60-198a-40ad-bb5d-9cfc8b5bfd3f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:55:58.888412 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-47309f60-198a-40ad-bb5d-9cfc8b5bfd3f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:55:58.900279 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47309f60-198a-40ad-bb5d-9cfc8b5bfd3f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:55:58.901870 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47309f60-198a-40ad-bb5d-9cfc8b5bfd3f service placement] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:55:58.911971 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1440/2919] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:55:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:56:01.971375 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4a94e7-072f-4e0c-bb24-b8934ad33c4e service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:56:01.984228 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4a94e7-072f-4e0c-bb24-b8934ad33c4e service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:56:01.992354 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c4f5f9-fb42-44fb-ace2-461a73b05ba4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:56:01.993025 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c4f5f9-fb42-44fb-ace2-461a73b05ba4 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:56:01.993308 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c4f5f9-fb42-44fb-ace2-461a73b05ba4 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:56:01.993607 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c4f5f9-fb42-44fb-ace2-461a73b05ba4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:56:01.995936 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-64c4f5f9-fb42-44fb-ace2-461a73b05ba4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:56:02.010353 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c4f5f9-fb42-44fb-ace2-461a73b05ba4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:56:02.011671 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c4f5f9-fb42-44fb-ace2-461a73b05ba4 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:56:02.026022 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1480/2920] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:56:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:56:08.907827 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=47 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:56:12.018547 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=51 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:56:34.161546 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47309f60-198a-40ad-bb5d-9cfc8b5bfd3f service placement] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:56:34.182697 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47309f60-198a-40ad-bb5d-9cfc8b5bfd3f service placement] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:56:34.188381 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68622fa7-8c93-4a28-bc5b-6a37f44b5422 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6143bddc05a46ea972709576c7ad95f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:56:34.189027 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68622fa7-8c93-4a28-bc5b-6a37f44b5422 service placement] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:56:34.189303 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68622fa7-8c93-4a28-bc5b-6a37f44b5422 service placement] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:56:34.189577 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68622fa7-8c93-4a28-bc5b-6a37f44b5422 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:56:34.191576 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-68622fa7-8c93-4a28-bc5b-6a37f44b5422 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:56:34.217632 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68622fa7-8c93-4a28-bc5b-6a37f44b5422 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:56:34.219236 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68622fa7-8c93-4a28-bc5b-6a37f44b5422 service placement] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:56:34.230151 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1441/2921] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:56:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:56:44.225296 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=15 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:57:45.477330 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c4f5f9-fb42-44fb-ace2-461a73b05ba4 service nova] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:57:45.505107 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c4f5f9-fb42-44fb-ace2-461a73b05ba4 service nova] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:57:45.511775 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47c471-9389-40e7-9080-dfe01b4d380e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:57:45.512446 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47c471-9389-40e7-9080-dfe01b4d380e service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:57:45.512732 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47c471-9389-40e7-9080-dfe01b4d380e service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:57:45.513015 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47c471-9389-40e7-9080-dfe01b4d380e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:57:45.515169 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cc47c471-9389-40e7-9080-dfe01b4d380e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:57:45.542983 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc47c471-9389-40e7-9080-dfe01b4d380e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:57:45.545130 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc47c471-9389-40e7-9080-dfe01b4d380e service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:57:45.558374 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1481/2922] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:57:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:57:47.928020 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68622fa7-8c93-4a28-bc5b-6a37f44b5422 service placement] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:57:47.941220 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68622fa7-8c93-4a28-bc5b-6a37f44b5422 service placement] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:57:47.946663 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46dba7c-ba5b-45d4-9f50-d5897e7eacca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e254671b21fe4661a381b9521810cf80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:57:47.947411 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46dba7c-ba5b-45d4-9f50-d5897e7eacca service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:57:47.947712 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46dba7c-ba5b-45d4-9f50-d5897e7eacca service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:57:47.947995 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46dba7c-ba5b-45d4-9f50-d5897e7eacca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:57:47.950210 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c46dba7c-ba5b-45d4-9f50-d5897e7eacca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:57:47.964930 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46dba7c-ba5b-45d4-9f50-d5897e7eacca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:57:47.966605 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46dba7c-ba5b-45d4-9f50-d5897e7eacca service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:57:47.978044 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1442/2923] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:57:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:57:49.361971 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47c471-9389-40e7-9080-dfe01b4d380e service nova] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:57:49.372109 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47c471-9389-40e7-9080-dfe01b4d380e service nova] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:57:49.378120 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6557ceb3-d601-45d1-b2c4-37763d6b8c57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2201e66caa4b4a57be09d845d7cb0023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:57:49.379105 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6557ceb3-d601-45d1-b2c4-37763d6b8c57 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:57:49.379487 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6557ceb3-d601-45d1-b2c4-37763d6b8c57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:57:49.379841 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6557ceb3-d601-45d1-b2c4-37763d6b8c57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:57:49.461658 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6557ceb3-d601-45d1-b2c4-37763d6b8c57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:57:49.463137 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6557ceb3-d601-45d1-b2c4-37763d6b8c57 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:57:49.483152 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1482/2924] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:57:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:57:52.390775 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46dba7c-ba5b-45d4-9f50-d5897e7eacca service nova] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:57:52.392289 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c46dba7c-ba5b-45d4-9f50-d5897e7eacca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:57:52.412070 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46dba7c-ba5b-45d4-9f50-d5897e7eacca service nova] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:57:52.420802 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745ee25-81ae-49ae-8cbc-226149fc2b0d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6143bddc05a46ea972709576c7ad95f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:57:52.421477 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745ee25-81ae-49ae-8cbc-226149fc2b0d service placement] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:57:52.421757 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745ee25-81ae-49ae-8cbc-226149fc2b0d service placement] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:57:52.422047 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745ee25-81ae-49ae-8cbc-226149fc2b0d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:57:52.424160 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6745ee25-81ae-49ae-8cbc-226149fc2b0d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:57:52.440922 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6745ee25-81ae-49ae-8cbc-226149fc2b0d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:57:52.443406 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6745ee25-81ae-49ae-8cbc-226149fc2b0d service placement] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:57:52.457318 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1443/2925] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:57:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:57:55.592723 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6557ceb3-d601-45d1-b2c4-37763d6b8c57 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:57:55.605315 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6557ceb3-d601-45d1-b2c4-37763d6b8c57 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:57:55.613197 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f4b477-929c-489e-bfb9-a2fe0f0ebe06 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6143bddc05a46ea972709576c7ad95f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:57:55.613879 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f4b477-929c-489e-bfb9-a2fe0f0ebe06 service placement] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:57:55.614166 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f4b477-929c-489e-bfb9-a2fe0f0ebe06 service placement] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:57:55.614481 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f4b477-929c-489e-bfb9-a2fe0f0ebe06 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:57:55.616642 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-06f4b477-929c-489e-bfb9-a2fe0f0ebe06 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:57:55.631273 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f4b477-929c-489e-bfb9-a2fe0f0ebe06 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:57:55.633365 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f4b477-929c-489e-bfb9-a2fe0f0ebe06 service placement] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:57:55.644918 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1483/2926] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 21:57:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:57:55.657667 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745ee25-81ae-49ae-8cbc-226149fc2b0d service placement] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:57:55.669515 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745ee25-81ae-49ae-8cbc-226149fc2b0d service placement] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:57:55.676457 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f8468f-8b63-4338-9c56-080479fd7492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:57:55.677250 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8468f-8b63-4338-9c56-080479fd7492 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:57:55.677531 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8468f-8b63-4338-9c56-080479fd7492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:57:55.677804 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8468f-8b63-4338-9c56-080479fd7492 admin admin] PATH_INFO: `/v3/users/f69162209070421faef53c4f75803f38` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:57:55.679112 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f8468f-8b63-4338-9c56-080479fd7492 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f69162209070421faef53c4f75803f38)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:57:55.681472 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f8468f-8b63-4338-9c56-080479fd7492 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:57:55.731069 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-70f8468f-8b63-4338-9c56-080479fd7492 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f69162209070421faef53c4f75803f38'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:57:55.732280 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-70f8468f-8b63-4338-9c56-080479fd7492 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f69162209070421faef53c4f75803f38'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:57:55.745812 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-70f8468f-8b63-4338-9c56-080479fd7492 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f69162209070421faef53c4f75803f38'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:57:55.754112 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-70f8468f-8b63-4338-9c56-080479fd7492 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f69162209070421faef53c4f75803f38'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:57:55.761656 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-70f8468f-8b63-4338-9c56-080479fd7492 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f69162209070421faef53c4f75803f38'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:57:55.768598 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1444/2927] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:57:55 2022] DELETE /identity/v3/users/f69162209070421faef53c4f75803f38 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:57:55.959850 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f4b477-929c-489e-bfb9-a2fe0f0ebe06 service placement] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:57:55.973732 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f4b477-929c-489e-bfb9-a2fe0f0ebe06 service placement] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:57:56.050277 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110538c7-bb61-49c7-ac8b-5eaf3276ef5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:57:56.051164 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110538c7-bb61-49c7-ac8b-5eaf3276ef5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:57:56.051478 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110538c7-bb61-49c7-ac8b-5eaf3276ef5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:57:56.051756 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110538c7-bb61-49c7-ac8b-5eaf3276ef5b admin admin] PATH_INFO: `/v3/projects/52d49b95be55483f93b8a325d6bff693` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:57:56.053136 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110538c7-bb61-49c7-ac8b-5eaf3276ef5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=52d49b95be55483f93b8a325d6bff693)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:57:56.055096 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110538c7-bb61-49c7-ac8b-5eaf3276ef5b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:57:56.095884 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:57:56.111933 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-110538c7-bb61-49c7-ac8b-5eaf3276ef5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52d49b95be55483f93b8a325d6bff693', 'request_id': 'req-110538c7-bb61-49c7-ac8b-5eaf3276ef5b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:57:56.118602 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-110538c7-bb61-49c7-ac8b-5eaf3276ef5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52d49b95be55483f93b8a325d6bff693', 'request_id': 'req-110538c7-bb61-49c7-ac8b-5eaf3276ef5b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:57:56.127637 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-110538c7-bb61-49c7-ac8b-5eaf3276ef5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52d49b95be55483f93b8a325d6bff693', 'request_id': 'req-110538c7-bb61-49c7-ac8b-5eaf3276ef5b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:57:56.162647 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-110538c7-bb61-49c7-ac8b-5eaf3276ef5b admin admin] The token cache is being invalidate because project 52d49b95be55483f93b8a325d6bff693 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:57:56.162950 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-110538c7-bb61-49c7-ac8b-5eaf3276ef5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:57:56.163785 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-110538c7-bb61-49c7-ac8b-5eaf3276ef5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:57:56.166099 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1484/2928] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:57:55 2022] DELETE /identity/v3/projects/52d49b95be55483f93b8a325d6bff693 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:57:56.171529 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f8468f-8b63-4338-9c56-080479fd7492 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:57:56.173123 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-70f8468f-8b63-4338-9c56-080479fd7492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:57:56.187156 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f8468f-8b63-4338-9c56-080479fd7492 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:57:56.244773 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14d18f9-5d5c-4f69-a864-9d958aa3ccf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:57:56.245543 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d18f9-5d5c-4f69-a864-9d958aa3ccf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:57:56.245818 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d18f9-5d5c-4f69-a864-9d958aa3ccf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:57:56.246085 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d18f9-5d5c-4f69-a864-9d958aa3ccf9 admin admin] PATH_INFO: `/v3/users/b982c54ed4ed47dc8cf55a2c34001db8` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:57:56.247307 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14d18f9-5d5c-4f69-a864-9d958aa3ccf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b982c54ed4ed47dc8cf55a2c34001db8)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:57:56.249365 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14d18f9-5d5c-4f69-a864-9d958aa3ccf9 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:57:56.295998 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a14d18f9-5d5c-4f69-a864-9d958aa3ccf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b982c54ed4ed47dc8cf55a2c34001db8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:57:56.296634 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a14d18f9-5d5c-4f69-a864-9d958aa3ccf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b982c54ed4ed47dc8cf55a2c34001db8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:57:56.297769 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:57:56.310933 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a14d18f9-5d5c-4f69-a864-9d958aa3ccf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b982c54ed4ed47dc8cf55a2c34001db8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:57:56.317776 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a14d18f9-5d5c-4f69-a864-9d958aa3ccf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b982c54ed4ed47dc8cf55a2c34001db8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:57:56.327804 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a14d18f9-5d5c-4f69-a864-9d958aa3ccf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b982c54ed4ed47dc8cf55a2c34001db8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:57:56.334351 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1445/2929] 10.222.0.65 () {58 vars in 1223 bytes} [Thu May 19 21:57:56 2022] DELETE /identity/v3/users/b982c54ed4ed47dc8cf55a2c34001db8 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:57:56.545763 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110538c7-bb61-49c7-ac8b-5eaf3276ef5b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:57:56.547395 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-110538c7-bb61-49c7-ac8b-5eaf3276ef5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:57:56.559889 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110538c7-bb61-49c7-ac8b-5eaf3276ef5b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:57:56.617706 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e47e9-2f97-4ea1-9be4-dccb0649a01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd45bbb3805754099870efea6f5d53dfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0436212ebbfa4ccb91d5457faaa851ee', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:57:56.618624 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e47e9-2f97-4ea1-9be4-dccb0649a01d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:57:56.618908 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e47e9-2f97-4ea1-9be4-dccb0649a01d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:57:56.619159 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e47e9-2f97-4ea1-9be4-dccb0649a01d admin admin] PATH_INFO: `/v3/projects/6154d8bf3c2f4119b0bdd36923a0137d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:57:56.620403 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e47e9-2f97-4ea1-9be4-dccb0649a01d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6154d8bf3c2f4119b0bdd36923a0137d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:57:56.622356 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e47e9-2f97-4ea1-9be4-dccb0649a01d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:57:56.678361 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f29e47e9-2f97-4ea1-9be4-dccb0649a01d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6154d8bf3c2f4119b0bdd36923a0137d', 'request_id': 'req-f29e47e9-2f97-4ea1-9be4-dccb0649a01d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:57:56.684493 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f29e47e9-2f97-4ea1-9be4-dccb0649a01d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6154d8bf3c2f4119b0bdd36923a0137d', 'request_id': 'req-f29e47e9-2f97-4ea1-9be4-dccb0649a01d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:57:56.690814 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f29e47e9-2f97-4ea1-9be4-dccb0649a01d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6154d8bf3c2f4119b0bdd36923a0137d', 'request_id': 'req-f29e47e9-2f97-4ea1-9be4-dccb0649a01d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:57:56.707924 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f29e47e9-2f97-4ea1-9be4-dccb0649a01d admin admin] The token cache is being invalidate because project 6154d8bf3c2f4119b0bdd36923a0137d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 19 21:57:56.708208 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f29e47e9-2f97-4ea1-9be4-dccb0649a01d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:57:56.708857 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f29e47e9-2f97-4ea1-9be4-dccb0649a01d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 19 21:57:56.713177 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1485/2930] 10.222.0.65 () {58 vars in 1232 bytes} [Thu May 19 21:57:56 2022] DELETE /identity/v3/projects/6154d8bf3c2f4119b0bdd36923a0137d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 21:58:06.330567 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=12,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:58:06.703408 dv-cinder-835635-2-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=68,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 22:01:39.148401 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14d18f9-5d5c-4f69-a864-9d958aa3ccf9 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 22:01:39.150043 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a14d18f9-5d5c-4f69-a864-9d958aa3ccf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:39.223669 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14d18f9-5d5c-4f69-a864-9d958aa3ccf9 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 22:01:39.289014 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6b82ac-c116-4687-b8c3-0dde5cbcb9c8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6143bddc05a46ea972709576c7ad95f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c4940befa9c4405699c72deedc8a2127', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 22:01:39.289667 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6b82ac-c116-4687-b8c3-0dde5cbcb9c8 service placement] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 22:01:39.289951 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6b82ac-c116-4687-b8c3-0dde5cbcb9c8 service placement] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 22:01:39.290193 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6b82ac-c116-4687-b8c3-0dde5cbcb9c8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 22:01:39.292265 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-fd6b82ac-c116-4687-b8c3-0dde5cbcb9c8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 22:01:39.304905 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6b82ac-c116-4687-b8c3-0dde5cbcb9c8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 22:01:39.306557 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6b82ac-c116-4687-b8c3-0dde5cbcb9c8 service placement] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 22:01:39.317803 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1446/2931] 10.222.0.65 () {60 vars in 1365 bytes} [Thu May 19 22:01:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 22:01:49.313596 dv-cinder-835635-2-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=58 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}